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ABSTRACT 



The purpose of this study Has to develop and 
evaluate procedures for validating a learning hierarchy from test 
data. An initial hierarchy for the computational skills of adding 
rational numbers with like denominators was constructed using Gagne’s 
task analysis. A test designed to assess mastery at each of the 11 
levels in this hierarchy was administered to a large sample of 
elementary school children. The pass-fail relationships from this 
test data were analyzed with seven learning hierarchy validation 
procedures, and seven hierarchical orderings of the 11 subtasks were 
determined. Fourth grade subjects, randomly assigned to seven 
treatment groups determined by the seven hierarchical orderings, 
worked 30 minutes a day on programmed materials until they were 
completed. Achievement tests measured acquisition of the terminal 
task. On the following day a transfer test on subtraction of rcitional 
numbers was administered. Two weeks later an alternate form of the 
achievement test was administered as a retention test. Results of the 
study indicate that sequence seems to have little effect upon 
immediate achievement and transfer. However, longer term retention 
seems quite susceptible to sequence manipulation. In general, the 
authors report that optimal instructional sequences can be devised 
using learning hierarchies validated from test data. (Author/JG) 
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Summary 



The purpose of this study was to develop and evalviate 
procedTires for validating a learning hierarchy from test data. 

An initial hieretrchy for the computational skills of adding 
rational numbers with like denominators was constructed using 
Gagne's task analysis. A test designed to assess mastery at 
each of the 11 levels in this hierarchy was administered to a 
large sample of elementary school children. The pass-fail 
relationships were analyzed using the following validation 
procedures adaptable for use with test data: (l) Pattern 

Analysis, ( 2 ) Guttman Scalogram Analysis, ( 3 ) the AAAS 
Approach, ( 4 ) Item Difficulty, ( 5 ) A correlational procedure 
developed using the phi coefficient, (6) Textbook Approach, 

(7) Randomization. Thus, 7 hierarchical orderings of the 
11 subtasks were generated. 

Programmed instructional materials were developed utilizing 
one lesson for each of the 11 subtasks. An evaluation of the 
efficacy of each validation procedure was conducted by actually 
sequencing the learning materials according to the hierarchies 
generated by each method and determining the effect of sequence 
i^on time to conqplete the program, achievement, transfer, and 
retention. Fourth grade subjects were randomly assigned to the 
7 treatment groups, defined by the 7 hierarchical orderings. 

Svd>jects worked independently throu^ the programmed booklets 
devotixig approximately 30 minutes per day to the materieUs 
until they were completed. Upon completion of the program 
each subject was given an achievement test measuring acquisition 
of the terminal task. On the day following ccmopletion of the booklet, 
a transfer test on subtraction of rational nuinbers was administered. 
Two weeks later an alternate form of the achievement test was 
administered as a retention test. An analysis of variance design 
was used to investigate the differential effects of sequence 
upon time. to complete the program, achievement, transfer, and 
retention. 

The results of this study indicate that the overall 
efficiency of the learning process can be affected by sequence 
manipulation. Sequence, even if random, seems to have little 
effect upon immediate achievement and transfer to a similar 
task. However, longer term retention seems quite susceptable 
to sequence manipulation. The mean retention score for the 
AAAS sequence grotp was significantly hi^er than those two 
of the other sequence grotqis. No significant differences were 
found across sequence groiq>s on the mean number of minutes used 
to complete the program. In general, the results suggest that 
optimal instructional sequences can be derived using learning 
hierarchies validated from test data. 



Introduction 



One of the major problems encountered by both teachers 
and authors of instructional materials is the sequencing of 
instructional activities (Hartung, 19 ^ 9 ', Hickey and Newton, 
1964; Gagne, 1967 ; Briggs, I968; Heimer, 1969)* Gagne (1963) 
stated that "the desi^ of an instructional situation is 
basically a matter of designing a sequence of topics." This 
statement suggests that there are optimal sequences of learn- 
ing events for the acquisition of a given terminal task. Do 
such optimal learning sequences exist? How are they deter- 
mined, and how are they verified? 

There is substantial evidence to support the general 
theory of the hierarchical structure of knowledge. The re- 
sults of several studies (Gagne and Brown, I96I; Gagne euid 
Paradise, I96I; Gagne, Mayor, Geurstens and Paradise, I962) 
sviggest that new skills and knowledge emerge from lower order 
knowledge, and that there is a significeuit amount of positive 
transfer from each successive subordinate level to the next 
hi^er level in a hierarchical ordering of such levels . In 
these studies Gagne and his associates did determine optimal 
learning sequences for different mathematical tasks . 

The sequence of subordinate tasks in a learning hier- 
archy, after sufficient validation, should describe a teach- 
ing program that will effectively accomplish the instructional 
objectives. That is, an instructional sequence based on the 
levels in the hierarchy will represent an optimal route for 
acquisition of the terminal task by a sample of learners. 

Empirical evidence supports an affirmative answer to 
the question, "Do optimal learning sequences exist?" Recent 
studies of sequencing (Niedermeyer, Brown & Sulzen, I969; 
Brown, 1970) indicated that the logical sequence group 
(based on learning hierarchies) performed reliably better 
than the scrambled group relative to time to complete the 
instructional program, errors made on the program, and errors 
made on a criterion test of complex problem solving skills. 
Task analysis, thou^ imprecisely defined, is a workable 
tool in the identification of the subordinate levels in a 
learning hierarchy. However, the question of validating 
the ordering of the levels in a hierar.chy is a much more 
complex and illusive problem. Gagne (l9^) stated that 
various methods have been tried but none seem entirely satis- 
factory as yet. 

Gagne's (1962) approach to hierarchy validation assumed 
that all lower level tasks, on which a higher level task is 
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dependent, must be mastered before tbe higher level task is 
DMistered. Validating a hierarchy based on this assumption 
is a tedious undertaking. The primary method employed in- 
volves the use of programmed learning materials. Using such 
miiterials, studies were conducted both in the laboratory 
with individual subjects and in classrooms, where groups of 
subjects responded to the leeurning programs. Numerous var- 
iations in the basic program w(5re investigated such as "high" 
or "low" guidance, "hi^" or "low" repetition and varying 
presentation orders of the frames. The proportion of posi- 
tive transfer to each higher-level knowledge from the rele- 
vant lower-level knowledges was calculated by the formula: 

4,. A + B 

= A + B + C ’ 

P+ = Proportion of positive transfer 

A = Number of learners passing both lower level 
and adjacent higher level tasks 

B = Number of learners failing both lower level 
and adjacent higher level tasks 

C = Number of learners failing lower level tasks 
but passing adjacent hi^er level task 

Perfect validity would be indicated by a ratio;, of 1.00. 

Gagne's work, after considerable revision, produced ratios 
of 1.00 or very near 1.00 and validated his hypothesized 
hierarchies . 

The hierarchical analysis vised by Gagne in association 
with the programmed learning sequences was adequate. How- 
ever, Phillips and Kane (1970) fovind this procedure to be 
inadequate when applied to test data. Previous exposvure of 
subjects to the subtanks of the hierarchy, confounded the 
issue of positive transfer. This study weus directed toward 
investigating the use of pattern analysis, scaling techniques, 
and correlational approaches in validating learning hierarchies. 
In brief, the purpose of this research was to develop and 
evaluate indirect procedures for validating a learning 
hierarchy from test data. It seems imperative that efforts 
to develop efficient procedures for validating learning 
hierarchies be undertaken. Gtigne's direct approach to learning 
hierarchy validation is valid but too tedious and costly to 
be undertaken in the cassroom or by publishers of commercial 
instructional materials. 
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An indirect vulidation proceduxe is defined as one based 
on the analyses of test data in contrast to a direct validation 
procedure based on the results of an instructional program. 

An outline of the steps undertaken to achieve these objectives 
is given below. 

Using Gagne's task analysis a learning hierarchy for the 
computational skills of rational number addition involving 
like denominators was constructed. Based on this logical 
ordering of the subtasks, a test was constructed to assess 
mastery at each level in the hierarchy. Tlie test was ad- 
ministered to 163 elementary school children in grades 4 
throxigh 6 to obtain a wide range of ability levels . The 
pass-fail relationships were analyzed using several indirect 
validation procedures including (l) Item difficulty (Nunnally, 
1967) , (2) The AAAS Approach (AAAS Commission on Science 
Education, 1968 ), ( 3 ) The Guttman Technique (Torgerson, 

1958) > (^) Pattern Analysis (Rimoldi and Grib, i960) and 

(5) Correlational analysis (described in the procedures section) . 

Based on the logically ordered hierarchy a programmed 
instructional sequence was developed utilizing one lesson for 
each subtask in the hierarchy. To test the efficacy of each 
of the indirect validation procedures , the programmed lessons 
were sequenced according to the hierarchies generated by 
each. Fourth grade subjects (l 42 ) were randomly assigned 
to the following treatments. 

(1) Logical Sequence - an ordering was based iq)on the 
sequence of sub tasks generated by a task analysis 
of the instructional objectives. No empirical 

' validation of this ordering was attempted.. The 

index of agreement (Rimoldi & Grib, i960) was 
calculated . 

(2) Item Difficulty - an ordering was determined by 
arranging the subtasks from simplest to most com- 
plex based on item difficulty. 

(3) Guttman Technique - an ordering was determined by 
the Guttman Scaling Technique. This procedure 
orders the subtasks such that the reproducibility 
coefficient is maximized. 

( 4 ) Correlational Analysis - an ordering was deter- 
mined by arranging the subtasks so that the 
correlation between adjacent tasks was maximized. 

(5) AAAS Approach - an ordering was determined by the 
AAAS hierarchy validation procedure . 
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(6) Textbook ordering - ein ordering was determined by 
arranging the subtasks in the usual textbook se- 
quence. This was accomplished by seqviencing the 
subtasks in an ordering which gave closest fit to 
that given in several elementary mathematics texts. 

(7) Remdom Sequence - an ordering was determined by 
randomly ordering the subtasks identified by task 
analysis. 

Subjects worked throvigh the programmed booklets inde- 
pendently devoting 30 minutes per day to the materials until 
they were completed. Unon completion of the program each 
s^^bject was given the same achievement test measuring 
acqiiisition of the terminal task. On the day following com- 
pletion of the booklet, a transfer test on subtraction of 
rational numbers was administered. Two weeks later a reten- 
tion test (alternate form of the achievement test) weus ad- 
ministered. An analysis of variance design was used to in- 
vestigate the differential effects of seqviencing on time to 
complete the program, achievement, transfer, and retention. 
Based on these results inferences w0re made concerning the 
adequacy of the indirect validation procedures for generat- 
ing an optimal learning sequence. 

Review of Related Research 

This study was concerned with the development and eval- 
uation of indirect procedures for validating learning hier- 
archies in mathematics. The research reviewed consists of 
studies (1) concerned with the sequencing of instructional 
materials in mathematics and (2) dealing with the psychometry 
of learning hierarchies . 

Sequencing Instructional Materials in Mathematics 

Instructional design in mathematics requires decisions 
about structuring the content and designing and ordering a 
set of instructional tasks. Gagne (1967) and Briggs (1968) 
have proposed the use of instructional sequences that re- 
quire the learner to follow a specific route through a con- 
tent structvire. Basic to this theory is the assumption that 
instructional sequence is most fruitfully formulated in 
cozijunction with content structure and that instructional 
sequence specifies the path the learner is to travel through 
the content structure. Thus in Gagne's learning hierarchy 
theory, task analysis is a major tool in designing instruc- 
tional sequences. 



Heimer (1969) indicated that are no clear guide- 

lines for the development of curricular materials. However, 
Gagne's theory and research pertaining to learning hierarch- 
ies and the sequencing of instructional materials have stimulated 
much research regarding sequencing variables. In the search 
for more definitive prescriptions for instructional design, 
seemingly contradictory results have been reported. Studies 
providing both evidence for and againii using rigorous methods 
of content sequencing are reviewed. 

Roe, CAse, and Roe (1962) conducted a comparative study 
of sequencing utilizing a item program on elementary 
probability. One group of students received a logically 
ordered form of the program, and one received a random ver- 
sion of it. A criterion test was administered to each sub- 
ject immediately upon cou5>letion of the program. No signif- 
icant differences were reported on time required for learning, 
errors during learning, criterion test score, or time re- 
quired for criterion test. In a similar study using an ex- 
tended version the same probability program. Roe (1962) 
found that the logically ordered sequence group performed 
significantly better on learning time, errors made during 
learning and on post learning measures. Roe (1962, p. 409) 
stated that "careful sequencing of items has a significant 
effect on student performance, at least for programs of some 
length €uad complexity." 

Levin and Bafcer (1963) reported a study in which a 60 
item geometry program for second graders was scrambled within 
20 item blocks . The results showed no significant differences 
in measures of acquisition, retention, or transfar between 
those who worked throu^ the logical program and those who 
completed the scrambled program. 

Payne, Ki/athwohl, and Gordon (1967, p. 125) stated that 
"no one seems to doubt, that were one to scramble a whole 
course that learning would be retarded, so that in part, the 
size of the unit in which sequence is destroyed is a factor . . . 
there may be a continuvan of dependence on sequence. At one 
extremis of the continuum, scrambling may have no effect on 
learning a set of spelling words which has no logical struc- 
ttcre- At the other extreme, scrsunbling would be expected to 
result in considerable decrease in learning if the leeu*ning 
of one concept were prerequisite to learning the next in a 
logical hierarchy." Payne et al. (1967) designed a study to 
test the foregoing assumption. They examined the effects of 
scrambling \q>on the learning of three programs. The three 
programs were ranked by trained, independent observers from 
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low to fairly high in logical interdependence. It was 
hypothesized tliat the effect of scrambling would he greatest 
for those programs dealing with tasks having the most logical 
development. The results of hctn immediate ajid delayed 
retention tests did not confirm this hypothesis. 

Miller (1965) conducted a study in which a 98 ~ft'*un® 
program on topics in ratio and proportion was presented in 
logical and random sequences to seventh graders. The author 
reported substantial differences in error rates which sup- 
ported the interdependency of the frames. The results, 
however, indicated that the scrambling of frames had little, 
if any, effect upon learning from the program. 

Hoi -la n d (1965) and Niedermeyer (1968) expressed concern 
over design and methodological weaknesses of the studies 
cited above. Holland (1965) pointed out that error rates 
for the two programs used in Roe et al. (l^ 2 ) did not differ 
significantly suggesting that the items were not interdepen- 
dent even in the logical sequence. Thus, no significant 
differences in criterion test scores should have been ex- 
pected. Holland claimed that the items vised in the Levin 
and Baker (1963) study were not hierarchical in structure. 
Further Levin and Baker suggested that the instructional 
materials used were relatively ineffective. Therefore, the 
results were not surprising. 

Niedermeyer (I968) pointed out that error rates in the 
programs used in the Payne et al. (1967) study were not 
substantially different. Again raising the question of 
whether any learning difference should have been esqpected. 

The authors pointed out another flaw in their design. Many 
of the subjects already knew a considerable amount of the 
material presented in learning sequence. Thus, any meaning- 
ful assessment of sequence effect on learning was difficult 
to obtain. 

lyatte (1969) indicated a major problem with studies 
conq>aring logical and random ordered sequences. It is diffi- 
ciilt to determine if the logical sequence is actually logical 
and the random sequence "random. " That is , that the random 
sequence is conpletely unbiased in the sense that none of the 
subtasks remain in the hypothesized hieraurchical ordering. 



SevereU. stxidies by Getgne and hi- associates, reviewed 
earlier, showed that when prograimned materials in mathematics 
were sequenced according to learning hierarchies, acquisition 
of hi^er level tasks occvurred only if all prerequisite sub- 
ordinate tasks were mastered. King (l 9 ? 0 ) described some 
key studies utilizing programs based on hierarchies and well 
controlled learning situations avoiding methodological weak- 
nesses pointed out in previous studies. These studies ore^ 
reviewed below. 

Using a program developed by Gagne and Brown (1961) 
for which there existed empirical data verifying a hi^ 
degree of interdependence among its concepts, Niedermeyer, 
Brown and Sulzen (1969) conducted a study of sequencing. 

The program consisted of 70 introductory frames ajid 4 o 
guided-discovery frames which required the learner to recall 
and use concepts learned in the introductory section. Tlae 
terminal objective of the program was for the learners to 
discover a formula for the sum of n terms of a series . 

Three groups completed a logical, scrambled, and reverse form 
of the program. The authors found that only the logical 
sequence group performed significantly better than a control 
group on both a test of concepts and a transfer test. The 
scranibled sequence group performed significantly better than 
the control group on the concept test, but not on the trans- 
fer test. No significant differences among the three se- 
quence groups were found on the posttests . 

Wodtke, Brown, Sands, and Fredericks (1967) administered 
a 74 -frame program on number bases in logical and scrambled 
versions. The hierarchical structure of the program was 
evidenced by the difference in error rates . On a pretest 
assessing foreknowledge of program material 90^ of the sub- 
jects scored zero. The mean posttest achievement score was 
18 on a 22 item test indicating the subjects actually leeurned 
from the program. However, there were no significant dif- 
ferences on achievement or aptitude- sequences interaction 
between those who studied the logical program and those who 
studied the scrambled program. 

Miller (1969) conducted a study using eight program 
sequences on matrix arithmetic. The results showed that the 
scrambled seqviences worked as well as the logical sequences 
for definitions and addition of matrices. However, in se- 
quences where subjects were forced to learn matrix multi- 
plication before learning definitions and matrix addition they 
perfv.nrmed significantly worse thaui those who learned needed 
definitions and matrix addition first. Miller concluded 



that mastery of individual tasks in a hiersirchy can be 
accomplished in several ways including a scrambled programmed 
sequence. However, a logical sequence still appears to be 
best in terms of overall effectiveness smd efficiency. 

In a recent study Brown (1970) found that logical se- 
quencing facilitated leeui’ning of programmed mathematical 
materials. Two sequence versions, logical and scrambled 
frame order, of the guided discovery program on nimiber series 
used by Gagne and Brown in their I96I study of concept forma- 
tion, were compared. The logical sequence group performed 
reliably better than the scrambled group relative to time to 
cGOoplete the instructional program, errors made on the pro- 
gram, and errors made on a criterion test of complex, prob- 
lem-solving skills . Brown concluded that when a sequence 
involves teusks that are complex problem solving behaviors 
that Gagne classifies as intellectual skills, ordering is an 
important factor in learning. Even for bright and relatively 
mature learners sequence can have an important effect upon 
learning . 

In summary, it appears that mastery of individual sub- 
teusks in a hierarchy can be achieved in several ways, 
including learning from randomly programmed sequences. 
However, as Miller (1969) pointed out logical sequencing 
still appears to be best in terms of overall efficiency and 
effectiveness. Several of the studies reviewed here suggest 
that varying sequences of instructional stimuli which have 
hig^ interdependency does not amke much difference in 
effectiveness of instruction. However, many of these studies 
are plagued with design problems. Thus, before the results 
of research can be applied to the problems of sequencing in- 
structional materials in mathematic .5 for use in the ordinary 
classroom situation, substemtial study of the effects of 
sequence upon time to achieve the terminal behavior, achieve- 
ment, transfer, and retention should be undertaken. 

The Psychometry of Learning Hierarchies 

Validating a learning hierarchy is not a simple under- 
taking. Many researchers (Ausubel, I963; Bruner, 1964; 

Gagne, 1965* Glaser, 19^4; and Suppes, 1966) have long re- 
cognized that sequence is a critical variable in learning. 

The learner begins with simple tasks and progresses to in- 
creasingly complex tasks. However, both Gagne (1968) and 
Pyatte (1969) have pointed out that determination of this 
hierarchical ordering of subtasks from simplest to most com- 
plex is a major problem. Various procedures for validating 



the' hypothesized ordering of the subordinate tasks in a 
learning hierarchy are reviewed below. 

Gagne and Paretdise ( 1961 ) were pioneers in learning 
hierarchy validation. Their approach was direct validation 
based on learners' responses to a programned learning 
Sfiquence and criterion tests administered immediately after 
the instructional program to establish pass-fail patterns 
for each component of the learning hierarchy. Consider the 
simple two-level hierarchy in Figure 1. 



Higher Level Task 
(more complex) 



Lower Level Task 
(less complex) 

Figure 1. A Simple Two-level Hierarchy 

Gagne's validation procedure was based on the assumption 
that teusk I must be mastered before task II can be mastered. 
Failxwe on task I would automatically produce failure on 
task II. Using + and - to represent pass ajid fail respect- 
ively, there are four possible pass-fail relationships which 
can be observed; (++), (+-), (—), (-+). For example, the 
first relationship signifies that the learner passed (per- 
formed to criterion ) both task I and task II. Only the re- 
lationship (+-) is in direct contradiction of the theory and 
indicates a fliw in ordering. The relationship (-+) (passed 
lower level task but failed higher level task) indicates a 
weakness in the instructional program but provides no infor- 
mation concerning the validity of the hierarchy. 

To validate a hierarchy Gagne analyzed the pattern of 
responses of each transfer in the hierarchy. That is, he 
constructed a contingency table of the observed responses 
to a higher level teusk and the task immediately prerequisite 
to it eu9 illustrated in Figure 1. He calculated the follow- 
ing ratio to determine the degree of validity of the hier- 
archy. 

Proportion of Positive Transfer (P+) = 

Perfect validity would be indicated by a ratio of 1.00. If 
all learners contradicted the theory, having observed 
patterns (+-) , then the ratio would be zero. Thvis, P+ is 
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bounded above and below by 1 and 0 respectively. The degree 
of validity of any hierarchy is measured by P+ with the 
lower limit of acceptability for P+ being .90* 

Phillips and Kane (l970) investigated the efficacy of 
this ratio when applied to test data alone. Using Gagne's 
task analysis a le€U*ning hierarchy for the computational 
skills of whole nxunber addition was constructed. Based on 
the hypothesized ordering of the subordinate levels, a test 
was constructed to assess mastery at each level. A second 
test utilizing a random ordering of the same items was con- 
structed. Both tests were administered to a large sample 
of elementary school children in grades 3 throu^ 6 in order 
to obtain a wide range of ability levels. The proportion 
of positive transfer between adjacent items on both tests 
was conputed using Gagne's formula. The proportions between 
adjacent items on both tests were above . 90 , except in two 
instances. Thus both the hypothesized and the random 
hierarchies were validated by this procedure. The authors 
concluded that prior educational experiences confounded the 
issue of positive transfer when considering test data alone. 



The Staff of the American Association for the Advance- 
ment of Science (AAAS Commission on Science Education, 1968 ) 
have refined and ex±ended Gagne's approach to hierarchy 
validation. Task analysis is used to generate hierarchies 
of subordinate subtasks. Learning seg.uences are designed to 
correspond to the hypothesized hierarchies . Pass-fail con- 
tingencies are used to test the dependency of each individual 
task on its immediate prerequisite subtasks. The Staff of 
AAAS pointed out that a high proportion of positive transfer - 
Gagne's statistics - is a necessary but not a sufficient 
condition for a valid hierarchy. Using the pass-fail rela- 
tionships defined by Gagne, the AAAS defined the following 
three ratios : 



( 1 ) 

( 2 ) 

(3) 



Consistency ratio » 
Adequacy ratio = 




Completeness ratio = 




Ratio ( 1 ) is a measure of how consistent the data are with 
the hyi^thesized dependency. Ratio ( 2 ) is a measure of the 
adequacy of the identified subordinate tasks. Ratio (3) is 
a measure of the effectiveness of instructional materials 



11 




i 



O 

ERIC 



designed to bring about learning. In the development of 
Science — A Process Approach , the AAAS has considered high 
consistency, adequacy, and completeness ratios as the neces- 
sary and siifTicient set of characteristics for a valid learn- 
ing hierarchy. No significance test has been developed for 
either Gagne's ratio or those defined by the AAAS. 

Cox smd Graham (1966) used the Guttman (l9^) Scalogram 
Analysis to develop a sequencially scaled achievement test. 
Essentially, the Guttman technique (Torgerson, 1958) orders 
items such that from knowledge of a learner's total score, 
his response pattern to the set of items can be predicted. 

The coefficient of reproducibility defined by: 

„ ^ total ninriber of errors 

“ “ total nuiriber of responses 

indicates the degree to which a set of items forms a perfect 
sceile. Error is defined as instances where a subject passes 
a hi^er level item after failing a lower level prerequisite 
item. Guttman suggested .90 as an acceptable lower limit 
for Rep. Cox and Graham reported a reproducibility coeffic- 
ient of .97 for their final arrangement of items and thus 
concluded their hierarchical ordering as valid. 

Several studies have been directed toward the validation 
of Bloom's Taxonomy (1956). Kropp, Stoker and Bashaw (1966) 
pointed out that the difficulty of the cognitive skills 
should iactease as the hierarchy is ascended. Since mastery 
of lower level tasks is reqiiired before mastery of higher 
.level more complex tasks, the difficulty of items should 
increase upwards along the hiereu'chy. Studies by Stoker and 
Kropp (1964) and Herron (19^5) showed that the cognitive 
skills in the Taxonomy did form a hierarchy. Items assess- 
ing skills higher up in the Taxonomy were more. difficult' 
than those at lower levels in the Taxonomy . Stoker, Kropp 
and Bashaw concluded that their results based on item 
difficulty validated the Taxonomy . 

There are several methods of hierarchical analysis 
reported in the literattire which are used in the generation 
of hierarchies rather than the validation of deductively 
analyzed hierarchies. McQuitty (i960) developed a procedure 
for determining if a hierarchical structure underlies a set 
of items. He began with a large item pool with no a priori 
assuniptions regarding the relationship between items in 
tezms of complexity. The procedure consisted of combining 
pairs of items or variables ^ich correlated highest with each 
other to fona new items. This procedvire is repeated until 
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one pair of items remains . When ita’:aS are platted on a 
linear scale and successive pairs axe connected by lines, 
the resulting diagram has a hierarchical structure. > Smith 
(1968) employed McQuitty's method to investigate the hier- 
archical model underlying Bloom's Taxonomy . In general, 

Smith conclvided that his analysis supported the Taxonomy 
rationale of a cumulative and hierarchical continuicn of 
cognitive processes. McQuitty (1966) has further refined 
and improved his method of hierarchical syndrome analysis . 

Multiple Scalogram Analysis (MSA) developed by Lingoes 
(1963) . can be vised to generate a hierarchical ordering from 
a large set of items for which there are no a priori assump- 
tions made regaxding order. The procedure essentially 
acconplishes the same goal as the Guttman Technique with 
built in controls against spuriously high reproducibilities 
as a function of e:ictreme marginal values. The MSA method 
involves selecting an item from a set to be analyzed, finding 
that item among the remaining items which is most like it, 
determining the number of eirors between the candidate item 
and all of its predecessors, and fimlly, applying a statist- 
2 

ical test (X ) of significance to adjacent item pairs . 
Whenever either the error or statistical criterion fails, 
the scale is terminated and another scale is started with a 
new item chosen from among those that remain, unti l the ite m 
set is esdiausted. This procedure tends to produce several 
branches which have very little in common with one another. 

Resnick and Wang (1969) have used MSA to generate 
various hierarchies. The methods developed by McQwitty 
(i960) and Lingoes (1963), as well as several other methods 
of hierarchical analyses outlined by Torgerson (1958) » are 
not readily adaptable for use in validating hypothesized 
hierarchies. In these methods the data must speak for them- 
selves with no a priori assumptions concerning order. 

Carroll (Resnick and Wang, 1969) has developed a hier- 
archy validation procedure based on the correlation between 
items or subtasks. This method, like those of Gagne and the 
AAAS, begins with the construction of pass-fall contingency 
tables for all possible pairs of items in the hierarchy. The 
phi-phimax coefficients are then computed for each table. 

Phi is the correlation between two adjacent dichotomous items 
(Nunnally, 1967) • Phimax is an estimate of the highest- 
possible phi coefficient given the marginals of the contin- 
gency table. G?he use of phimax in the denominator controls 
against artificial inflation due to extreme pass-fail rates, 
since this value becomes larger sus the pass-fail rate becomes 
larger. This procedure is most useful in empirical searches 
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for hiereu:chical relationships among large quantities of 
data. In discussing different approaches to hierarchy vali- 
dation, Resnick and Wang (1969) cited no studies utilizing- 
Carroll's method. 

Grib and Rimoldi (i960) developed a procedure for com- 
paring two patterns of responses from a number of subjects 
on a number of items . Responses to items may or may not be 
dichotomously scored. Listing each subject's responses to 
a set of items produces an observed matrix of responses with 
rows corresponding to subjects and columns corresponding to 
items. An expected matrix can be formed based on an opera- 
tional definition of what response patterns are expected 
from a given set of items. The only restriction on the 
e 35 >ected matrix is that the subjects total score on the ex- 
pected pattern must equal his total score on the observed 
pattern. These two patterns can be compared and an index 
showing the amount of agreement between the two patterns can 
be computed lasing the following equations. 

When the e 3 q>ected matrix of ones and zeros (l = pass; 

0 = fail) haa been generated weights are calculated for each 
cell of the expected pattern. For the cells containing ones, 
the weights a.j are given by 



R. C. 

2 R. 

1 



, where 



,th< 



R. “ number of ones in the i row. 

1 

C. “ number of ones in the J column. 

J 

For the cells containing zeros, the weights a^. are given by 

R. C. ^ 

^ i j , where 

®-iJ 

R. number of zeros in the i row.: 



1 

‘til 

Cj = nuinber of zeros in the J column. 

The index of agreement I , indicating the amount of 

Cl 

agreement or correlation between two patterns, is given by 
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, where 






T - 



m^t 



T = sum of all a. . axLd a. . 

ij ij 



A = svun of a. . and a . . of the cells that are the same in the 
t ^ J 

observed and expected patterns. » 

A. = sum of a. . and bT~. corresponding to minimum possible agreement, 
m t ij ij 



The above formula is used when comparing patterns across 
subjects. A similar formula cam be derived for comparing 
patterns across items. The index of agreement varies between 
zero (no agreement) emd one (perfect agreement) . Grib and 
Rimoldi compared values of to the coefficients obtained 

using the pattern analysis developed by Green (1956) and 

fovind I gives conservative values. The authors report no 
a 

significance test for the index of agreement. 



In sunanary, Gagne's procedure for validating a learning 
hierarchy using programmed learning sequences is adeqiiate 
and iiseful in research designed to gain empirical evidence 
to support the hierarchical structure of knowledge. However, 
for validating specific hierarchies to be used as guidelines 
for the sequencing of classroom instructional activities , 
less e3Q>ensive indirect procedures using test data should be 
developed. 

Several types of hierarchical analysis (McQuitty, 196O; 
Lingoes^, 1963 J Torgerson, 1958) are viseful in generating 
hierarchies from a large item pool with no a priori assump- 
tions regarding the relationship between items in terms of 
con^>lexity. In these methods the data must speak for 
themselves with no a priori assumptions concerning order . 
Thus, these procedures are not readily adaptable for use in 
vaulting hypothesized hierarchies. 



It seems that hierarchical analysis such as the Guttman 
Technique (Torgerson, 1958) , Pattern Analysis (Rimoldi & 

Grib, i960) and the AAAS Approach (AAAS Commission on Science 
Education, 1968) are quite adaptable for validating deduct- 
ively analyzed hierarchies from test data. Other procedures 
based on the correlation between test items or levels of the 
hierarchy and the difficulty of items could be useful in 
validating a hierarchy from test data. The present study 



was designed to test the adequacy of each of these procedures 
for validating a learning hierarchy from test data by se- 
quencing instructional materials according to the hierarchy 
generated by each and determining the effect of sequence 
upon achievement y transfer, retention and time to complete 
the instructional sequence. 

Development of Materials and Procedures 

The major theme of the present research was the differ- 
ential effects of sequencing instructional programs accord- 
ing to a learning hierarchy validated by various procedures 
upon (l) time required to complete the program, (2) achieve- 
ment, (3) transfer, and (4) retention. Will one sequence 
produce maximal achievement, facilitate the greatest amount 
of transfer and retention, and require less time to complete? 
This research was designed to answer these questions . Based 
upon the answers inferences were drawn concerning the 
adequacy of the indirect validation procedures used in gen- 
erating each instructional sequence. 

In order to accomplish effectively the objectives of 
this study, experimental decisions had to be made concerning 
the appropriate terminal task for the hierarchy and the 
sample of learners to participate in the experiment. These 
concerns are discvissed below. 

1. Terminal Task ; The time of the ye«u: in which the 
study was conducted influenced the choice of the terminal 
task to be used in developing the learning hierarchy. Sub- 
jects were to work through the instructional materials in 
April. This meant a terminal task had to be selected \rhich 
the subjects had not previously been exposed to yet at the 
same time ensuring that the learner had the necessary back- 
ground for achieving the instructional objectives. Throu^ 
the aid of textbooks and suggestions from elementary school 
teachers, the addition of rational numbers involving like 
denominators was chosen as the terminal task. 

2. Samples ; First, a sample of subjects for collecting 
data to be used in the hierarchy validation had to be se- 
lected. In order to obtain a wide range of ability levels, 
tests were administered to 163 elementary school children 

in grades 4 through 6. The fourth graiders included in the 
sample were high achievers who had begun studying addition 
of rational numbers . Such a range weis chosen so that both 
correct and incorrect responses were obtained at all le'irels 
in the hierarchy. Second, a sample of learners to work 
through the instructional material was selected on the basis 
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of two pretests . These pretests vrexe designed, to assess 
entering behaviors . Pretest I was designed to determine if 
the learners had acquired the necessary prerequisites for 
mastering the skills in the learning sequence. Pretest II 
was designed to weed out those learners who haid already 
acquired the skills to be taught in the learning sequence . 

In this manner, lk2 fourth graders were selected to participate 
in the experiment. 

Throughout this report the reader will encounter some 
terms which may be unfamiliar . Operational definitions of 
these terms are given here. 

1. Optimal Learning Sequence ; An optimal learning 
sequence was defined as one which maximally facilitates 
achievement, trauisfer, and retention. 

2- Valid Hierarchy ; A valid hierarchy was defined as 
one which yields an optimal learning sequence. 

3 • Adequate Indirect Validation Procedure ; An evalua- 
tion of the indirect validation procedures was made by actu- 
ally sequencing instructional materials according to the- 
hierarchies generated by each validation procedvire. The 
hierarchy yielding the best approximation of an optimal 
sequence, as defined above, was considered valid. The ade- 
quacy of each indirect validation procedure was measvired by 
the degree to which a hierarchy validated by the procedure 
yielded an optimal learning sequence. 

Development and Analyses of the Hierarchy 

Using Gagne's task analysis a learning hiereurchy for the 
computational skills of rational number addition was constructed 
The sequence of subtasks generated was reviewed by four 
authors of elementary mathematics texts. Based vq>on this 
formative evaluation of the adequacy and completeness of the 
hierarchy, the sequence of subtasks was revised until, to 
e^qperts in mathematics education, there were no obvious 
flaws in the learning hierarchy. The hypothesized hierarchy 
is shown in Appendix B. 

Based on the hypothesized ordering of the subordinate 
levels, a test was constructed to assess mastery at each 
level in the hierarchy. The test was designed to minimize 
chance or careless errors. A procedure of test construction 
similar to the "H-technique" (Stouffer, Borgatta, Hays, and 
Henry, 1952) was adopted. The test consisted of composite 
test items for each level in the hierarchy. Each composite 
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item consisted of three items testing the same subordinate 
task. Pass at each level was defined as correct responses 
to at least two of the three items for the level. A sample 
test item for level VII is given in Figure 2; the entire 
test is given in Appendix D. 

4 1/9 1 3/8 13 5/12 

+5 2/9 +2 l/8 + 6 3/12 

Figure 2. Sample Test Item 

The entire test consisted of 11 composite items making a 
total of 33 items. The internal consistency of the test was 
determined using the Kuder-Richardson Formula 20 (Nunnally, 

1967). 

The test was administered to 163 elementary school 
children in grades 4 throu^ 6 to obtain a wide range of 
ability levels. It was administered by the classroom 
teachers and was conrpleted by all subjects in one sitting. 

The test was not a timed test. Subjects were instructed to 
atteoopt all items and were given sufficient time to do so. 

The pass-fail relationships were analyzed using various 
indirect validation procedures. These procedures are 
described below. 

Pattern Analysis 

The pattern analysis technique developed by Rimoldi 
and Qrib (1980) was used to analyze the responses for the 
con^lete hierarchy on a subject by subject bMis. As prev- 
iously described, the index of agreement I indicates the 

amount of agreement or correlation between two patterns. In 

this case, I was calculated between the observed and ex- 
& 

pected patterns of pass-fail relationships. 

The items on the test were sequenced according to the 
hypothesized ordering of the subordinate levels in the 
hierarchy. If this ordering was truly hierarchical, where 
each subtask was a necessary prerequisite to the next, once 
a learner failed a given level he would be expected to fail 
all subsequent levels. Thus the expected pattern was defined 
as one where no correct response followed an incorrect re- 
sponse. Using 1 AJTi^ 0 to represent pass and fail respect- 
ively, a hypothetical observed matrix and the corresponding 
expected matrix are given in Figure 3. The responses of 
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each subject were recorded on punch cards . A program waus 
written vrtiich transformed each subject's observed pattern of 
responses into the escpected pattern as indicated in Figure 3- 
Wei^ts for each cell and the index of agreement were calculated 
using the formulas given earlier. 
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Expected Pattern 



Figure 3« Sample observed and corresponding expected matrices. 
Item Difficulty 

The pattern of responses to the 11 items were analyzed 
to determine if the items were hierarchically ordered in 
terms of item difficulty. Required mastery of certain lower 
level necessary prerequisite subtausks before the next higher 
subtask cam be mastered implied that the numiber of leaurners 
psasing a lower level task must be greater thaui or equal to 
the numiber passing the next higher level task. In other 
words if the items were atrranged hieraurchically frcm. simplest 
to most complex y the first item would be mastered by most 
lecurners tested. Each successive level or item would be 
more difficult in terms of the necessity of more recaJLl from 
preceding prerequisite items. Therefore, the number of 
learners passing each hi^er level item would decrease. The 
difficulty of any dichotoomis item (p-value) is the fraction 
of learners tested who passed that paurticular item or level 
(NunnaiUy, 1957) • For instamce, a p-value of .90 would snean 
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that 90 percent of the learners teste''^. passed the item. 

Thus, the observed p-values for the levels of the hierarchy 
should form a decreasing sequence of values. The maximum 
p-va''ue should occur at the lowest level in the hierarchy and 
the .a-f nimum p-value should occur at the terminal tausk. 



As previously stated, the test items were arranged 
according to the logically constructed hierarchy for the 
addition of rational numbers . A hierarchical chart of the 
11 items or levels was drawn and the p-value for each was 
taken from an item analysis and entered in the hierarchy. 

This hierarchy of observed p-values was examined to deter- 
mine if the expected pattern was exhibited. That is, the p- 
values should be higher at the lower levels of the hierarchy 
and decrease to a m1 nimum at the terminal task. Serious 
deviations between the observed p-value rank and the expected 
p-value rank would indicate that the items were not hierarch- 
ically ordered. 



It should be noted that validation based on p-values 
could only be made by comparison. There was no statistical 
test for significance of difference. One must appeal to his 
knowledge of the subject matter at hand rather than rigorous 
statistical procedxires. For instance, suppose the following 
three items were hypothesized as being hierarchically ordered. 



I. 



4 



II. 



i4 

+23 



III. 27 

m 



Further, svppose the observed p-values for items I, II, and 
III were .94, .87 and .89 respectively. Since item III in- 
volves renaming and item II doesn't, item III would not be 
placed before item II based on this slight difference in 
p-values. However, if the difference was considerably greater, 
one would have to reorder the items and assume item II to be 
more difficult than item III. Also, the p-value at any level 
can be affected by simply constructing a more difficult item 
testing the same subtasks. However, this problem can be 
greatly controlled by careful test construction. Despite the 
crxuleness of this procedure, it did provide a simple test 
for determining whether group performance , as a whole, vali- 
dated the hierarchy in terms of predicted p-values. 



Hii Coefficient 



Gagne (1968, p. 3) stated, "A learning hierarchy repre- 
sents the most probable expectation of greatest positive trans- 
fer for an entire sample of learners concerning >diom we know 
nothing more than what specifically relevant skills they 
start with." In order to empirically validate a hiaxsarchy 
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of subordinate aubtaoks which x>ossei>Ged this characteristic, 
a procedure based upon the correlation between the levels 
was devised. If the 11 test Items (levels in the hierarchy) 
were arranged hierarchically, there should be high correla- 
tion between adjacent items. Ordering the items based on the 
correlations between them would allow prediction of a 
learner's success on any given item based upon his performance 
on previous lower level items. For instance if a learner 
passed Items 1 and 2 and item 3 correlated highly with item 2, 
the learner wovild be expected to pass item 3. On the other 
hand if the learner failed items 1 and 2, he would not 
be expected to pass item 3. This approach to hierarchy 
validation, based on the phi coefficient is similar to the 
Fhl/Fhimax procedure used by Carroll (Resnick & Wang, I969) . 
The correlational procedure used in this st\idy is outlined 
below. 



The product -moment correlation of two dichotomous 
distributions or test iteiis is called "phi" (Nunnally, I967) . 
To illustrate the usefulness of the phi coefficient in 
assessing the degree of hierarchical relationship between 
two test items (or levels of a hierarchy) , the definition of 
phi is given in terms of the pass-fail relationships shown 
in the contingency tables in Figure 4 . 



Item 1 



Item 1 



pass 

Item 2 

fail 



fail pass 



IT 


TT — 







Table a 



pass 

Item 2 

fail 



fall pass 



— g 


a 


c 


d 



Table b 



Figure 4 . Pass-fail contingency tables. 

Table a of Figure 4 indicates the four possible pass-fail 
relationships when a learner encoimters two test items or 
subtasks of a learning hierarchy. That is, the learner may 
(1) fail item 1 and pass item 2; (2) pass both items 1 and 2; 
(3) fail both items 1 and 2 or; ( 4 ) pass item 1 and fail 
item 2. Sjrmbolizing the four quadrants as shown in table b 
of Figure 4 , phi is defined as follows: 

" (a + b)(c +“'4]|(b + c)(a + b) 

By examination of the above formula, one sees that if 
the number of subjects falling in q\iadrants a (++) or c (— ) 
is zero, then phi is negative. If an equal number of 
subjects fall in each of the four quadrants, then phi is 



zero. If all subjects tested fall in one of the four quad- 
rants (leaving zeros in the remaininf^ three), the numerator 
becomes zero yielding a zero phi -value. The larger the pro- 
portion of learners falling in either quadrants a or c 
becomes (short of N, the total number of learners tested), 
the larger phi becomes. Thus in order to obtain meaningful 
results in terms of hierarchical analysis, careful attention 
must be given to the data used in ceU-culating phi. That is, 
ozie must ensure that a sxifficient munber of learners fall 
in each of the four quadrants of the contingency table b in 
Figure 4 , Eind at the same time guard against too great a 
proportion of the learners tested having either the relation- 
ships (++) or (— ) . If the majority of the learners have 
achieved (or not achieved) both items, then phi is artific- 
ial -1y inflated or decreased, sheding no light upon the hier- 
archical relationship of the two items. There are at least 
three ways in which this problem can be controlled to some 
escbent . 

1 . When \ising the phi coefficient to indicate the 
hierarchical relationship between items, test a 
large saas>le of learners representing a wide range 
of ability and achievement levels. This will en- 
sure a greater balance of the number of learners 
falling in each of the four pass -fail categories. 
Thereby reducing the artifact. 

2 . Censor the data. That is, define operational rules 
pertaining to the data used. Control of the number 
of learners falling in each of the four quadrants can 
be exercised by eliminating data from specific 
learners. For instance, if some learners have al- 
ready mastered too many or too few of the items do 
not include these data in the analyses. 

3 . Use some procedure such as Carroll's phi-phimax 
method. Phimax is an estimate of the highest- 
possible phi coefficient given the marginals of 
the contingency table. Since phimax would become 
larger as the pass or fail rate of either items 
became more extreme, the use of phimax in the de- 
nominator controls e^alnst artificial inflation of 
the index due to extreme pass or fail rates. 

The first control method was used in this study for 
three reeusons: (l) It is difficult to develop an adequate 

algorithm for discarding data and even more difficult to 
defend. Such a formula would almost have to be defined in 
terms of a given sample, (2) a search of the literature pro- 
vided no evidence in support of the Carroll 3nethod. No 
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enipiriceU "tryout" of the efficacy ox this procedure was 
reported, (3) due to the wide range of ability levels tested, 
it was felt that extreme pass or fail rates would have mini- 
mal inflationary effects upon the phi coefficient. 



In order to further exemplify how phi was used to vali- 
date the hierarchy vised in this stvidy, an exaiiple is given. 
Svtppose that five items (levels in a hierarchy) are hypo- 
thesized as being hierarchically ordered 1, 2, 3, 4, 

Item one is defined as the easiest item on the test or 
lowest level in the hierarchy. Phi is calculated between 
item 1 and each of the other four items. 



^ (1.2) ^ (3.2) ^ (2.4) 

^ (1.3) (3.4) 0 (2.^) 

0 (1.4) 0 (3.5) 

0 (1.5) 

Hypothesized ordering Ordering based on phi 
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3 

4 

5 



1 

3 
2 

5 

4 



Figure 5. Hypothetical Ordering Via Hii. 

Suppose ^ ^ between items 1 and 3) is the largest. 

Now phi is calculated between items 2, 4, ^ and the largest 
phi underlined. The process is repeat k-2 times, where k.'.ls 
the nuniber of items. The resulting hierarchy generated by 
phi is given in Figure 



In a hierarchy with k levels there are k! possible 
orderings. Ordering the levels via phi seeks the ordering 
which maximizes phi between adjacent levels as described 
above. If there are no a priori assunptions made concerning 
the hierarchical ordering the levels, then phi must be cal- 
culated between item 1 and the remaining k-1 levels choosing 
the hi£^est phi and repeating the process k-2 times as shown 
in Figure 5. However, in validating a deductively analyzed 
hierarchy, there are a priori assumptions made concerning 
transfer from one item (level) to another. In Figure 6, the 
clusters of items, in which hi^ positive transfer between 
adjacent items was expected, are shown. 
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2/9 + 3/9 
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l/lO + 3/10 + 5/10 
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2 + 1/2 
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31/7+2 3/7 
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41/9+5 2/9 
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31/6+4 1/6 + 2 2/6 
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7/4 = 
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3 5/8 + 2 7/8 




11. 


72/5+4 1/5 + 2 4/5 
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Figure 6. I^pothesized clusters of items having hi^ correlations 
between adjacent items. 



This procedure for ordering items by the phi coefficient was 
applied within these clusters of items. Of the k! orders 
within each cluster, the one which maximized the phi coeffic- 
ient between adjacent items was chosen as the hierarchical 
ordering for that cluster. This procedure lead to the 
hierarchiccU ordering of all 11 items. 

The formula given here for phi cau be derived as a 
special case of the usual product -moment formula. When 
correlating two dichotomous distributions or items, exactly 
the same results is obtained from phi that would be obtained 
from the product -moment formula. Thus, in order to utilize 
existing computer programs, phi values were calculated from 
the product-moment formula. No statistical test of differ- 
ences between the phi's was used. Decisions concerning 
ordering of items were based strictly upon the numerical 
values of phi. 

The Guttman Scalogram Analysis 

The Guttman Scalogram Analysis (Torgerson, 1958) was 
used to detennine the extent to which the 11 items could be 
arranged in an order such that passage of a certain item 
reliably predicted passage of all items lower in the hier- 
archy. A hypothical set of perfectly scaled items is shown 
in Figure ?• 
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Figure 7* A perfect Guttman Sca,le 

Subjects are listed dawn the side; items are listed along 
the top. Note that once a subject fails an item all subsequent 
items are failed. Similarly, if a learner passes a given 
item, he has passed all prerequisite items. Thus from 
knowledge of a learner's total score, his response pattern 
to the set of items can be predicted or reproduced. The 
proportion of responses to the items that can be correctly 
reproduced is a measure of how well a set of items can be 
ordered such that the response patterns form a triangular 
euray as in Figure 7. The proportion of responses to the 
items that can be correctly reproduced is defined as the 
coefficient of reproducibility (Rep) . 



Rep = 1 - 



Total number of errors 
Total number of responses 



Error is defined as a case where a subject passes a higher 
level item after failing a lower level item. For instance, 
if one is scaling 4 items, patterns like (-+++), ( — ++), 
(+-+-) represent 1, 2 and 1 errors respectively. The value 
of .90 was suggested by Guttman (1944) as an acceptable 
lower limit for Refp. 

Since hierarchically ordered items should exhibit the 
triangular pattern of a perfect Guttman sce^.e, the Guttman 
scaling technique should be of some value in hierarchy 
validation. From a high Rep (> .90), the hierarchical re- 
lationship of the entire set of 11 items could be inferred. 
Thus, the object was to arrange the 11 items in such an 
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order so as to maximize Rep . A compater program for the 

Cornell technique of scalogram analysis (Guttman, 194?) was { 

employed. The program analyzed the pass -fail patterns of 
subjects to the 11 items and formed permutations of the 

ordering of these until Rep was maximized. The print-out i 

gave the optimal ordering of the items and the Rep for the 
optimal ordering. 

} 

AAAS Procedure { 

The AAAS Connnission on Science Education (I 968 ) pro- 
cedTires for veO-idating a learning hierarchy were moiLfied for 

use with test data to investigate the dependency of each 

individual subtask on its immediate prerequisite. The AAAS 

approach has been used successfully when Tising test data in ) 

conjunction with instructional materials . This approach is . ! 

outlined below. 

Consider a sin^jle two-level hierarchy with a terminal | 

task and two subordinate subtasks (Figure 8 ) . If subjects 

TERMINAL TASK \ 



Subordinate Subtask Subordinate Subtask f 

. ) 

Figure 8 . Sinqple Two-Level Hierarchy 

are asked to respond to test items assessing mastery of each i 

of the three cells of this hierarchy, there are 8 possible 
patterns of performance outcomes which can be observed. 

Using the code 0 and 1 for fail and pass respectively, the 8 1 

possible configurations are: 

11110000 

1 1 , 0 1 , 1 0 , 0 0 , 1 1 , 0 1 , 1 0 , 0 0 j 

The asswqption being that for those learners passing the 
terminal task, it is highly probable that they are also able 
to pass the two subordinate subtasks. 

J 

In Table 1, the patterns of zeros aiul ones are translated 
into descriptions of mastery of the terminal task and mastery 
of the subordinate subtasks. I 

Table 1. Possible pass-fail patterns in a two-level hierarchy. 



(++) 


(+-) 


(-+) 


(-) 


1 


111 


0 


0 0 0 


1 1 


0 1 , 1 0 , 0 0 


1 1 


0 0 , 0 1 , 1 0 





The relationships, (++) passed botji the terminal task and the 
subordinate subtasks, (+-) passed the terminal task and 
failed one or both subordinate subtasks, (-+) failed the 
terminal task and passed the subordinate subtasks, (— ) 
failed the terminal task and failed one or both subordinate 
subtasks, are represented by the four columns. 

The following three ratios are defined in terms of 
these relationships. 

(1) Consistency ratio. = 

The consistency ratio is defined as the quotient of the num- 
ber of patterns consistent with the hypothesis divided by 
the total number of subjects who acqiiired the terminal task. 
The value of this ratio is a measure of how consistent the 
data are with the hy]pothesized dependency. 

(2) Adequacy ratio = ■ 

The adequacy ratio is defined as the quotient of the number 
of patterns consistent with the hypothesis divided by the 
sum of the number of subjects acquiring both the terminal 
and subordinate tasks and the number of subjects acquiring 
the subordinate subtaaks but not the terminal task. The 
adequacy ratio is a measvire of the adequacy of the identified 
subordinate tasks. 

(3) Con5>leteness ratio = 

The completeness ratio is defined as the quotient of the 
total number of patterns consistent with the hypothesis 
divided by the sum of the number consistent with the theory 
and the nuoiber of subjects failing both the terminal and 
subordinate tasks. The completeness ratio is a measure of 
the effectiveness of instruction. 

In Science - A Erocess Approach high ( .90 or above) 
coiisistency ratio, adequacy ratio and completeness ratio for 
every dependency relationship are considered the necessary 
and sufficient set of characteristics for a valid hierarchy. 
However, when attenpting to validate a deductively analyzed 
hierarchy from test data alone, the above procedure must be 
modified. In both the AAAS procedure and Gagne's work, the 
relationships (-+) (lower level passed, hi^er level failed) 
suggests that there may be something inadequate in the 
instructional material related to acquiring the terminal 
task and that these relationships should not be included in 
a measure of support for the hypothesis. However, when 
analyzing test data this relationship does support the hier- 





27 




archical nature of the subtasks. With test data, there is 
no basis for esipecting a learner to pn.ss a superordinate task 
■when all subordinate tasks are passed as in an instructional 
program. The learner may simply have reached his achievement 
le'vel. One would expect, howe-ver, that once a learner failed 
aji item he would fail all subsequent hi^er level items. 

Thus with test data alone only (+-) of the four possible 
relationships between adjacent items contradicts the hier- 
archical theory. 

The consistency ratio does indicate how consistent the 
test data are. ■with the, hypothesized ordering. A low con- 
sistency ratio bet^ween t'wo adjacent levels •would indicate a 
problem in ordering since this ratio is 1.00 (perfect) if no 
one contradicts the theory and decreases as the nmber of 
subjects contradicting the theory increases. A low adequacy 
ratio may simply indicate that a large proportion of 
learners tested reached their achievement level at this 
point in the hierarchy. If this ratio is too low it would 
indicate that a wider range of ability and achie^vement le^vels 
should be included in the saniple tested. The relationship 
( — ) is in accord with the theory when using test data. 

That is, no higher le^vel ■bask is passed after failure on the 
lower level tasks. Th\xs, the completeness ratio is of very 
little valvie in detezmining the hlerarchiceil ordering of 
task from test data. Of course, this ratio is of great 
importance when using data from an instructional program. 

In stannary, when ■validating a hierarchy from test data 
the consistency ratio is of major importance. An acceptable 
le^vel for this ratio in the present study -was set at .85. 

In the AAAS ■work the level of acceptability ■was .90. How- 
ever, when analyzing the dependency of adjacent items from 
test data alone it seemed reasonable to accept a lower ■value 
as evidence of a ■valid hierarchical ordering. The adequacy 
ratio indicates, more than anything else, a flaw in sampling. 
Since the relationship (-+) is in accord ■with the hierarchi- 
cal theory when using test data, it was not expected or 
necessary for this ratio to be as hi^ (-90) an that recom- 
mended in the AAAS report. An acceptable le^vel for this ratio 
in the present study was set at . 70 . Again, the completeness 
ratio when using test data indicated a flaw in sampling and 
not a flaw in ordering. Since the relationship (— ) is in 
accord with the hierarchical theory and many subjec^ts at the 
upj)er le^vels of the hierarchy ■would be expected to fall in 
this category, an acceptable level for this ratio ■was set at 

.50. 



In using the modified AAAS;; procedure to validate the 
hierarchy used in this study, the items were arranged so as 
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to maximize the consistency ratio ana to keep the adequacy 
ratio at an acceptable level. No attempt was made at con- 
trolling or maximizing the completeness ratio. In addition 
to computing the three ratios of the AAAS procedure, the 
proportion of positive transfer used by Gagne was calculated. 
Another ratio defined by the authois as the order ratio was 
computed. The order ratio indicated the proportion of 

order ratio = { I ' lj I •«• “ (+-) 

learners' response patterns which were consistent with the 
theory. The level of acceptability for this new ratio., was 
set at .90 as a lower limit. 

Textbook Sequence 

The 11 instructional lessons were integrated and se- 
quenced according to the "usual textbook ordering." The 
"usual textbook ordering" was determined by examining differ- 
ent elementary mathematics texts. The followiaag three fourth 
grade texts were used in determining this ordering: 

(1) Deans, E., Kane, R. B., McMeen, G. H., A 
Oesterle, R. A. Understanding Mathemtics . 

New York: American Book Company, I9S8. 

(2) Duncan, E. R., Capps, L. R., Dolciani, M. P., 

Quast, W. G., & Zweng, M. Modern School 
Mathematics : Structure and Use . New York; 

Hougihton Mifflin Companj'-, 19^7. 

(3) Keedy, M. L., Dwi^t, L. A., Nelson, C. W., 

Schluep, J., & Anderson, P. A. Exploring 
Elementary ^thematics . New York: Holt, 

Rinehart & Winston, Inc., 1970. 

The order of presentation of the 11 subtasks in these 3 texts 
was very siioilar. The textbook sequence was determined by 
arranging the subtasks in an ordering ^ich gave closest fit 
to that given in these texts. 

The subtasks were, of covo:se, not presented in zts iso- 
lated and pure form as in the instructional materials de- 
veloped. Often, 2 or 3 of the subtasks were presented in 
one section or lesson. Also, addition and subtraction of 
rationals were presented aimost simultaneously. Thus in 
order to keep the program length the same for comparative 



purposes and have the lessons sequenced in the "osiial text- 
hook ordering", the logically ordered lessons were integrated 
to form new lessons. That is, one o^ more of the lessons 
were combined to form a new lesson. However, the same type 
and number of responses were required of the student. This 
sequence was used in order to gain some useful information 
in answering the question, "Do authors of instructional, 
materials need to he concerned with more rigoro\is and logi- 
cal organization of suhtasks within chapters or sections?" 

Random 

A random ordering of the 11 items (suhtasks) was 
included for two reasons . First , an instructional sequence 
based on a random ordering of the lessons could serve as a 
control group in the analysis of variance model. Second, 
coBiparison of the random group with each of the other groups 
co\ild provide useful information in determining if an optimal 
ordering of the 11 items was acMeved. Research evidence on 
the sequencing of instructional activities is somewhat con- 
tradictory. Several studies yield results which support the 
theory of careful sequencing while others yield results that 
suggest, that careful sequencing has little effect upon 
learning. 



PeveloTament of Instructional Materials 

A programmed text format was chosen for two reasons: 
first, it could provide uniform instruction for students in 
different groups. Thus variance due to sources not of pri- 
mary interest in this research could he reduced. Second, 
this format allowed complete random assignment of subjects 
to treatment groups. In order to conduct this study in the 
public schools, intact classes heid to he utilized. The use 
of programmed instructional sequences permitted subjects, 
rather than classes, to he randomly assigned to treatment 
groups. Thus, strengthening the genevalizahility of the results. 

The instructional materials consisted of an eleven lesson 
programmed booklet on the addition of rational nimibers 
with like denomimtors (Appendix C) . The program utilized 
one lesson for each of the 11 levels in the learning hier- 
archy. The procedure used in writing these lessons en^ployed 
techniques similar to those of Crowder and Skiimer. The 
method employed was similar to what May (1965) called hybrid 
programming. Conventional exposition and problem sets were 
coinbined with the Skinner mode. Each of the 11 lessons was 
designed to develop the specific skill represented by the 
coxrresponding hierarchy level. The lessons were from 2 to 
3 pages in length making a total program length of 29 pages. 

The program called for frequent responses from the learner 
which he wrote in blanks provided. A page from the booklet 
is given in Figure 9 to illustrate the format used. 




30 



34 , 



\ 



AKSUnS 



i 



U«« %Im ahidad per^i^ms to h»lp writ* • Mix«d nuMral 
that nviMS th» •mm rwnber &a 11 • 




11 V- 

8 a 



□.Q. 

a 8 



g » i 

CoMplM* th« foUcndjng 



... -a9-o9 



(b> 



U - 2 +CD - {^ LwJ I« 2 in lommt 

9 9 9 9 9 

2.Q 



tenH>? 



9 3 



Thu» 1Z _ 

9 ■ 3 






« j..,.9-op.oip 



Ml I. 



Um thi tliMM portloM to help you write a Biaed mawral 



that iMMiS the 



•r •• 2 

4 




B wtr — I 



□ n I — 1 

, LJl t L==! 

4 4 4 4 



.□._.p.oa 



o 

ERIC 



Figure 9» Sample instructional lesson. 
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The first draft of the instructional sequence was 
written with the lessons (frames) sequenced according to the 
logically constructed hierarchy. The lessons were written 
as concisely and briefly as possible while still achieving 
the instructional objectives of the lesson. This was done 
to Iceep the program at a realistic length and to minimize the 
reading load. 

The first draft was reviewed by three authors of ele- 
mentary mathematics texts. Judgments were made concerning 
the appropriateness and correctness of language , a,dequacy 
and effectiveness of art work, layout and format, and in 
general the overall expected effectiveness of the leetrning 
sequence in teaching the skills of rational number addition. 
Based upon the sviggestions of these authors, the materials 
were revised. 

Since the subjects were accustomed to using commercially 
produced materials, steps were taken to make the learning 
sequence as attractive in appeareince as possible. The use of 
ditto masters was selected for reproducing the materials. 

This method allowed the use of colors, easy correction of 
typographical errors, and a relatively simple procedure for 
producing the needed art work. All drawings were in red and 
green and the textual print was in blue. Lessons were re- 
produced on 8 1/2 X 11 inch paper with each page having a 
2 inch answer column at the right. Each lesson (frame) 
began on a separate sheet indicating major segments of the 
program and allowing for easy reordering of the lessons for 
each treatment grovq>. 

In order to assess the effectiveness of the learning 
program, a tryout was conducted utilizing five fourth grade 
students. Pre- and post-test achievement was measured by an 
18 item test. The pretest mean was I.8O; the posttest mean 
was 13.60. The difference between the means was significant 
at the .01 level. Note was taken as to where these students 
had difficulty in understanding terminology, intezpreting 
drawings and exaitiples, and following the exposition. Based 
on these observations, the materials were revised again. 

Based on thesre suits of this tryout and the inspection by 
experienced authors, the materieils were judged as adequate 
for achieving the instructional objectives. The lessons 
were ordered according to the hierarchies generated by the 
seven indirect validation procediires. The lessons and pages 
of each sequence were then appropriately numbered. From the 
exterior all booklets apx>eared the same. They seemed adequate 
in terms of attractiveness, durability, and usability for 
fourth grade students. 
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Experimental Procedures 

Fourth grade subjects were selected to participate in 
the study on the heusis of two pretests. Pretest I (Appendix 
D) was designed to determine if the learners had mastered 
the necessary prerequisites for successfully achieving the 
skills presented in the programmed text. The test concen- 
trated on the concept of fraction, recognizing parts of a 
whole, reading and writing fractional numerals, 'v^ole number 
addition, and simple whole number division. Pretest I was 
administered to 175 fovirth grade subjects one week prior to 
initiation of the learning sequence. The students were at 
two different schools in the same Indiana covinty. Subjects 
were grouped according to ability. Each school had one class 
of hi^, medixan, and low ability students. Pretest II was 
administered only to those students judged, on the basis of 
pretest I, ready to undertake the programmed materials. 

Pretest II (Appendix D) was designed to determine if the 
students had already mastered the skills to be taught in the 
instructional sequence. The test consisted of one item for 
each of the 11 levels in the hierarchy. Pretest II was an 
alternate form of the achievement test administered at the 
completion of the instructional program. Only those subjects 
judged, on the basis of pretest II, to have mastered an in- 
significant nuBiber of the skills in the instructional pro- 
gram were inclvided in the study. Of the 175 students tested, 
l42 met the criterion on both tests. Thus, the sample of 
learners participating in the study knew the necessary pre- 
reqtiisite skills and knew, essentially, none of the skills 
presented in the learning sequence. In order to control any 
changes in behavior dxie to time elapse, pretest II was ad- 
ministered on Friday of one week and the programmed se<#ience 
began on the following Monday. 

Since the students were already stratified on ability, 
subjects were assigned randomly to treatment groups within the 
three strata high, medium, and low ability. This procedxire 
strengthened the unbiasedj^ss of the samples since essentially 
the same number of high, medium, and low ability students 
were in all treatment grox 5 >s. By the principle of randomiza- 
tion all treatment grovq)s were considered equal. 

All 175 fourth grade students at Rossville Elementary 
School, Rossville, Indiana and Kyger Elementary School, Frankfort, 
Indiana, completed the programmed instructional, booklet. Of 
this number, only i42 stxidents were actually included in the 
study. Thirty did not setfcisfy the criteria on pretest I and II 
and three were lost due to experimental mortality. 



The students -worked through the prograjnmed booklets 
independently devoting approximately 30 minutes per day to 
the materials xmtil they were completed. The experimenters 
explained how students were to use the materials and assisted 
the students in any problems they encountered for the first 
t-wD days of the study. Folio-wing this, the teacher super- 
-vised the students’ -work until completion of the study. 

The students entered their responses to questions 
directly in the booklets using a cardboard cover-up for the 
answer column. Subjects -were instructed to keep the answer 
column covered until they entered their response then pull 
the cardboard down to reveal the correct response. If their 
response was correct, they were instructed to continue to the 
next question and repeat the process. If their response -was 
incorrect, they were instructed to draw a line throu^ the 
incorrect response and enter the correct response above it. 
Before moving to the next frame, students were advised to 
attempt -to determine why their response -was incorrect and 
the authors' response was correct. Much of the time, their 
mistakes were due to carelessness and they could see them. 
However, if they could not, they were instructed to ask the 
teacher for help at this point. Teachers were instructed to 
give help only in the context of each child's material. For 
instance, if one child's sequence had a frame -which involved 
-writing the simplest name for fractional numerals but no 
preceding frame dealt. with the definition of simplest name or 
the mechanics of renaming, the teacher did not show the 
student the manipulations invol-ved in renaming. The students 
were guided in using only the information and art -work provided 
in the gi-ven frame. This procedure led to considerable 
fioistration on the part of some students in trea-tment groups 
-vdiere necessary prerequisites were not pro-vided. Ho-wever, 
teachers had to adhere to this procedure if the effects of 
sequencing iQ)on achievement, retention, and transfer -were 
to he measured. 

The booklets were distributed by the teacher at the be- 
ginning of each work session. However, due to other school 
activities the 30 minute work session length varied somewhat 
from day to day. The teachers kept a log book of the exact 
number of minutes spent on the booklets each day. As each 
child finished his booklet, the date and time of completion 
-was entered in his booklet. At the completion of each 
session, the booklets -were collected by the teacher. No 
student was allowed to take his booklet home or st-udy it 
other than during the siipervised work session. In order to 
ke^ students from rushing through the materieds, they were 
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reminded at the beginning of each Eedsion that they were to 
study the materials and try to remember what they did, not 
just copy in the correct responses. They were also told 
that they would be tested upon completion of the booklet. 

For the duration of the study, the students had no mathematics 
instruction other than these work sessions. Teachers provided 
other mathematics activities for each student who finished 
his booklet until the whole group had finished. 

An achievement test measuring acquisition of the terminal 
behavior was administered to each student on the day following 
completion of the programmed sequence. The test consisted 
of two items per hierarchy level except for two levels which 
pertained to renaming. An alternate form of this test was 
administered as a retention test two weeks after the administration 
date of the achievement test. During this two week period, 
students studied mathematical topics other than operations 
with fractions. A transfer test on subtraction of rational 
numbers with like denominators was administered on the day 
following administration of the achievement test. This test 
consisted of 10 items analogous to those covering the rational 
number addition. No renaming in the subtraction process such 
‘ as required in 3 l/8 - 5/8 was included on the test. However, 

as on the addition test, students were required to reduce 
I answers to lowest terms. The achievement, trsuasfer euid retention 

[ tests are included in Appendix D. 

j Analysis of variance for multiple groups, unequal n's 

model, (Winer, 1962) was used to investigate the differential 
effects of sequencing on each of the four dependent variables 
achievement, transfer, retention, and time to determine if 
the meaui scores of the 7 sequence groups differed significantly. 

I Other statistical procedures used are discussed in the next section. 

Results 

This research was directed toweurd the development and 
evaluation of procedures for validating a leetrning hierarchy 
from test data. An evaluation of the efficacy of each pro- 
cedtire was conducted by actually sequencing learning materials 
according to the hierarchies generated by each method and 
determining the effect of sequence upon achievement, transfer, 
retention, and time to complete the program. In order to 
accomplish the objectives of this study, various methods of 
I hierarchical analyses were used. Before r^orting the results 

1 . of seqtience effects, the results of these hierarchical 

analyses are presented. 

! 
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Hierarchical Analyses 



A test designed to assess mastery at each level in the 
hierarchy for addition of rational numbers was administered 
to 163 elementary school children in grades 4 throiogh 6 
(Appendix B) . The internal consistency (Kuder-Richardson 
Formula 20 ) of the test was .81. The pass-fedl relationships 
were analyzed by several scaling techniques and methods of 
hierarchical analyses, adaptable for use with test data, to 
generate hierarchical orderings of the 11 levels. The 
results of these hierarchical analyses are presented below. 

Pattern Analysis 

To provide evidence that the logical ordering was in- 
deed logical, the pattern analysis technique developed by 
Rimoldi and Grib (i960) was used. The index of agreement 
was .87. No reordering of the 11 items was attempted. 

Item Difficulty 

Validating a learning hierarchy using item difficulties 
is based on the assimiption that items at the lower level of 
the hierarchy are single and easy and that they get in- 
creasingly more complex and difficult moving up the hierarchy. 
The observed p-values for the original ordering of the 11 
items in Table 2. Tables 2 through 3I, in which statistical 
results are displayed may be found in Appendix A. Inspection 
of these values indicated that items 3 and 7 were out of 
order. That is, there were items above them in the hierarchy 
which had greater p-values indicating they were easier items. 

The hierarchical ordering generated from the observed 
p-values is shown in Table 3 - The items were rearranged so 
that the p-values formed a decreasing sequence of values. 

Phi Coefficient 



The results of the validation procedure using the phi- 
coefficient described previously are shown in Tables 4 
through 6. Three clusters of items were hypothesized as 
having high dependency upon one another. The correlation 
matrix for the items within each cluster was calculated. ^ 
inspection of these matrices, the items within clusters 
were arranged so that the correlation between adjacent items 
was maximized. The correlations between all adj6u:eztt items 
in the validated ordering were significant at the .01 level. 
The resulting hierarchical ordering of the 11 items is shown 
in Table 7. 
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Guttman Technique 



Student responses to the 11 items were analyzed using 
Guttman Scalogram Analysis. This procedure was used to 
determine the extent to which the 11 items could be arranged 
in an ordering such that passage of a certain item reliably 
predicted passage of all items lower in the hierarchy. The 
proportion of responses to the item that can be correctly 
reproduced is a measure of how well a set of items can be 
ordered such that the response patterns form a trianguletr 
array of a perfect Guttman Scade. This proportion is defined 
as the coefficient of reproducibility (Rep) . 

The pass-fail patterns of subjects to the 11 items 
were analyzed and permutations of the ordering of these were 
formed until Rep was maximized. The revised ordering yielded 
a Reproducibility Coefficient of -9^. The hierarchical 
ordering validated by the Guttman Technique is given in 
Table 8. 

AAAS Procedure 

The modified AAAS procedure described earlier was used 
to measure the dependency of each item on its immediate 
prerequisite. The items were arranged so that the consistency 
ratio was maximized. The consistency ratios between adjacent 
items of the revised ordering are listed in Table 9- All of 
these values, except one, were greater than or equal to .85. 

The adequacy ratios for the revised ordering of the 11 
items are listed in Table 10. This ratio indicated, more 
than anything else, a flaw in sampling. Since the relationship 
(-+) is in accord with the hierarchical theory when using 
test data, it was not ejected or necessary for this ratio 
to be as hi^ (*90) as that recommended in the AAAS report. 

All of the values in Table 10 were greater than or equal to .70. 

The completeness ratios for the revised ordering are 
listed in Table 11. Ko attempt was made at controlling or 
maximizing this ratio. All values in Table 11, except one, 
were greater than or equal to .50. 

The proportions of positive transfer and the order co- 
efficients as defined by Gagne and the authors respectively 
are listed in Tables 12 and 13 • Gagne's notion of propor- 
tion of positive transfer was modified for use with test 
data to derive the formula for the order ratio. This ratio 
indicated the proportion of the subjects' response patterns 
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which were consistent with theory. The level of accept- 
ability for this ratio was set at .9^'j as a lower limit. 

All values listed in Table 13, except one, were greater than 
or equal to .90. 

Textbook Ordering 

Three elementary school mathematics texts were examined 
to determine this ordering. The order of presentation of 
the 11 subtasks in these 3 texts was very similar. In 
examining these texts, it was found that all subtasks per- 
taining strictly to renaming skills were presented in the 
chapter preceding operations with rational numbers. None of 
the 11 subtasks were treated in as isolated form as they 
were represented in the hierarchy. For instance, adding 
with two fractions having like denominators and adding with 
three such fractions was presented as one unit. Adding with 
mixed numerals (either 2 or 3 addends) which did or did not 
require reducing the fractional parts to lowest terms were 
all treated together. 

All 3 texts presented addition with mixed nTamerals 
where the sum of the fractional parts is greater than 1 as 
the moat complex skill in addition of rational numbers with 
like denominators. However, such additions involving 2 and 
3 addends were all treated together. 

There were slight differences in how the three texts 
presented the 11 siibtasks in the hierarchy. The tesctbook 
ordering used in the present study was determined by choos- 
ing an ordering which gave the closest fit to that presented 
in the 3 texts. The ordering resulting from this procedure 
is shown in Table l4. In more than one instance, two or 
more of the subtasks were integrated to form new subtasks. 
Thus, the textbook ordering contained only 7 subtasks. The 
instructional material for this sequence, however, was the 
same length in terms of student responses as for the other 
sequences . 

Random Ordering 

The 11 subtasks of the hypothesized hierarchy were 
randomized to form this ordering. The resulting ordering 
is shown in Table 15 along with the orderings generated by 
the other procedures. 

The index of agreement was calculated for each of the 
7 orderings. These values are given in Table l6. The 
index of agreement for the random ordering was .61. 
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Pretests 



Siibjects to participate in the experimental stxidy of 
the effect of sequence on achievement, transfer, retention, 
and time to complete the program were selected^on the hajsis 
of their performance on two pretests. These tests were de- 
signed to assess entering behavior . Pretest I was designed 
to answer the question "Does the learner have the necessary 
prerequisite skills needed to master successfully the skills 
presented in the learning program?" Pretest II was designed 
to answer the question "How many of the skills presented in 
the learning program has the learner already mastered?" 

Some of the items on Pretest I were jvidged more crucial 
than others. However, in no instance was a learner accepted 
who responded incorrectly to more than 7 items. Thus, an 
acceptable score was defined as one ranging between 24 and 
17 on a 24 point test. The proportion of subjects obtaining 
each acceptable score on Pretest I are listed in Table I7. 
Note that 67 % of the subjects included in the study obtained 
scores of 20 or higher with 23 % having perfect scores. Only 
13^ of the subjects gave incorrect responses to 5 or more 
items on Pretest I. The mesux score on Pretest I was 21.84. 

Pretest II consisted of 11 items assessing mastery at 
each of the 11 levels in the hierarchy upon which the learn- 
ing program was based. The proportion of subjects obtaining 
each acceptable score is given in Table I8. Note that 71^ 
of the subjects tested were unable to respond correctly to 
any of the 11 items. Ninety-two percent of the subjects 
gave correct responses to 2 or less of the 11 items. The 
mean score on Pretest II was O.563. 

One hundred forty-two subjects met the pretest criteria. 
That is, their scores were between 17 and 24 on Pretest I and 
between 0 and 4 on Pretest II. 

The Effects of Sequence 

The major ptn^ose of the present research was an invest- 
igation of the differential effects of sequencing instruc- 
tional materials according to a learning hierarchy validated 
by various procedures upon time to complete the program, 
achievement, transfer, and retention. This research was de- 
signed to answer the question, "Will one seqiience require 
less time to complete and maximally facilitate achievement, 
transfer and retention?" The results of the effect of se- 
quence upon these four variables are presented below. The 
internal consistency coefficients for all tests used appear 
in Table 19. 



Achievement 



An achievement test displaiyed in Appendix C, was administered 
one day following completion of the instructional program. 

Of major interest was the comparison of the logical sequence 
derived using Gagne's task analysis, and the random, sequence 
groiq> with all sequence groups. Planned comparisons (Hays, 

1963) were made between the mean of the logical sequence 
group on the achievement test and the means of all other 
sequence groups. No significant differences, at the .0^ 
level, were found between the mean achievement score of the 
logical sequence group and the other sequence groups. 

Similarly, no significant differences were found between the 
mean achievement score of the random sequence grovq> and the 
other groups. 

The differential effects of sequence upon achievement 
were investigated using an analysis of variance design. 
Identification of the groups is given in Table 20. The one- 
way analysis of variance on achievement is shown in Table 21. 

No overall significant differences were found. 

Transfer 



A transfer test (Appendix D) on the subtraction of 
rational numbers was administered on the day, following com- 
pletion of the achievement test. Planned comparisons between 
the mean transfer score of neither the logical nor random 
sequence groups with all other groups showed any significant 
differences at the .05 level. The one-way analysis of 
variance on transfer is shown in Table 22. No overall signif- 
icant differences were found. 

Retention 



An alternate form of the achievement test administered 
at the completion of the program was administered two weeks 
after conq>letion of the learning program as a retention 
test. Using planned comparisons the mean retention score of 
the logical sequence group was compared with all other means. 
The difference between the logical seqvience group mean (7. 50) 
and the textbook sequence group mean (4.9^) was significant 
at the .05 level. AH other comparisons were nonsignificant. 
Similarly, compeurisons between the random sequence group 
mean and all other means were not found to be significant. 
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The one-way ajmlysis of variajicj on retention is shown 
in Table 23. The F-ratio of 2.12 was very near the critical 
valvie 2.15 for significance at the .05 level. Thus, post-hoc 
coBDparisons of all means using the Duncan Multiple Range 
Test (Winer, 19^2) were made. The difference between the 
means of the AAAS sequence group (8.52) and the textbook 
seqxzence group (4.95) was statistically significant at the 
.05 level. Also the difference between the means of the 
AA^ sequence group and the item difficulty sequence group 
(5*37) was statistically significant at the .05 level. All 
other comparisons were nonsignificant. 

Time 

Planned comparisons between the mean number of minutes 
to complete the programmed instructional booklet of the 
logical and random sequence groups and all other groups were 
made. At the .05 level, only one of these comparisons was 
statisticelly significant. Namely, the difference between 
the mean time required to conplete the program by the logical 
sequence group (103*86) suid the correlational sequence group 
(135.33). The range of the number of minutes required by 
each group to complete the instructional program is listed 
in Table 24. 

The one-way analysis of variance on time is shown in 
Table 25. No overall significant differences were foxmd at 
the .05 level. 

Further Investigations of Sequence Effects 

Responses were first marked as incorrect if they were 
not written in sin5»lest form. Responses such as 9/l2 or 
3 11/8 were considered incorrect since they were not written 
in simplest form. EIxamination of the tests revealed that 
many students scoring very low had actually mastered the 
skills involved in rational number addition. However, due 
to not following directions or having not mastered the renaming 
skills subjects failed to write the answers in siii5>lest form. 

In order to assess more accurately the differential effects 
of sequencing, two other scoring procedures were \ised. Too 
many low scores d^^e solely to not renaming could obscure the 
true effects of sequencing. A second scoring algorithm 
gave one-half credit for responses which were ,:correct but 
not reduced to lowest terms. A third scoring algorithm 
disregarded renaming. 
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The one-way analyses of variance on achievement allowj.ng 
partial credit and disregarding reducing to lowest terms in 
scoring aie shown in Tables 26 and 27 respectively. No over- 
all significant differences were fotmd in either case. 

Planned comparisons between the mean achievement scores of 
the logical and random sequence groups and all, other means 
revealed no significant differences in the means at the .05 
level. 

The one-way analysis of variance on transfer allowing 
partial credit in scoring is shown in Table 28. The differ- 
ence between treatment means was not significant at the .05 
level. Coaiparisons of the logical and random sequence group 
means with all other group means showed significant differ- 
ences between only two pairs of means . The difference in 
the means of the random sequence group (6.89) azid the phi 
coefficient sequence group (4.76) was statistically signifi- 
cant at the .05 level. A significant difference was found 
between the means of the random sequence group (6.89) and the 
textbook sequence group (4.52). 

The one-way analysis of variance on transfer disregard- 
ing reducing to lowest terms in scoring is shown in Table 29. 
The F-ratio was significant at the .05 level. The Duncan 
Multiple Range Test indicated significant difference between 
two pairs of means. The differences between the means of the 
random sequence group (8.26) and both the phi coefficient 
sequence group (5-10) and the textbook sequence grovq> (5 •19) 
were significant at the .05 level. 

The one-way analyses of variance on retention allowing 
partial credit and disregarding reducing to lowest terms in 
scoring are shown in Tables 30 and 31 respectively. The 
F-ratio in the first case was near the critical val\ie for 
significance at the .05 level. The Duncan Multiple Range 
Test indicated significant differences between two pairs of 
means. The AAAS sequence group mean was significantly 
greater than that of both the item difficulty and the text- 
book sequence groups. 

When disregarding reducing to lowest terms, the F-ratio 
was significant at the .05 level. Significant differences 
between two pairs of means were fovmd using the Duncan test. 
The AAAS sequence group mean was significantly greater than 
those of both the item difficulty and the textbook sequence 
groups. 
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Discussion 



The reader's attention was directed toward two troublesonu? 
problems with many sttidies of the effects of sequence reported 
in the literature. First, in comparing the effects of a 
logical and a random sequence upon learning, it was not 
demonstrated that indeed a logical sequence and an unbiased 
random sequence were being used. Second, in many of the 
studies reported, it was suspected that too many of the sub- 
jects aJLready knew much of the material presented in the 
learning program. This study was designed to mirrimize the 
possibility of these pitfalls. 

The index of agreement was used to determine if the 
hypothesized ordering developed through the use of task 
analysis was indeed logical.. That is, that it was hier- 
archical in structure. The index of agreement was .87 
which indicated that the observed response patterns of the 
subjects correlated hi^ly with the expected patterns in- 
dicating that the logical ordering was logical. The index 
of agreement for the random order was . 61 . Thus, the logical 
ordering appeared to have markedly more of the characteristic 
of hierarchical structure than did the random ordering. 

With the exception of the textbook ordering all other sequences 
were validated enpiricaliy using varioxis procedures. The 
indices of agreement for all validated orderings were above 
.83 indicating high correlations between observed and 
expected response patterns. The index of agreement for the 
textbook ordering was only .62, however. 

Subjects included in the st\xdy had to meet stringent 
criteria on two pretests. Namely, they had to have the 
necessary prerequisites for undertaking study of the skills 
presented in the instructional program, and they could not 
have already mastered the skills to be taxight. Thus, the 
probability was very low that outcomes attributed to sequence 
were affected by the aforementioned problems. 

On the other hand due consideration must be given to 
two sources of artifact over which we had less control than 
would have been desirable. 

1 . Teachers were instructed on the type and amount of 

student help to provide. However, they reported that 
learners in the random sequence group, who were asked 
to perfoim certain tasks when they had not mastered 
necessary prerequisites, were very frustrated. In 
these instances, the teachers may have provided too 



much instruction mahing assessment of sequential 
effects difficult. This could have accounted for 
the absence of significant differences among the 
mean achievement scores of the seven sequence groups. 

2. Examination of subjects' responses revealed that 
many students did not write the answers in lowest 
terms. Again, the teachers were instructed to 
stress directions and be s\ire all learners under- 
stood what was expected of them. Thtus, it might 
be concluded that the lessons pertaining to reducing 
to lowest terms were not adequate in terms of 
allowing for enou^ practice and repetition. How- 
ever, when allowing partial credit or disregarding 
reducing to lowest terms in scoring, still no 
significant differences were found on immediate 
achievement . 

Neither planned nor post hoc comparisons showed any 
significant differences between the logical sequence group 
and the other sequence groups on achievement, transfer, or 
retention. The logical sequence group did require signifi- 
cantly less time to complete the program than did the correla- 
tional sequence group. This suggests that careful task 
analysis of instructional objectives can be a powerful tool 
in devising optimal instructional sequences. In fact it may 
mean, in terms of overall cost, that careful analysis of 
the instructional objectives to reveal the prerequisite sub- 
tasks is an adequate procedure for developing a valid hierarchy. 

Conclusions 



Within the limitations of this study the results seem 
to jtistify the following conclusions: 

1. The overall efficiency of the learning process, 
using programmed instructional materials, can be 
affected by changing the sequential ordering of the 
subtasks . 

2. Sequence, even if random, has little effect upon 
ixmoediate achievement . 

3- Retention appears to be the variable, of the four 

xmder study, most susceptable to sequence manipiilation. 



4. No sequence maocimally facilitated achievement, 
retentxon, and transfer, and reqtiired less time to 
complete. However, based on the group means, the 
AAAS procedure yielded the best sequence overall. 

5 . Textbook authors may need to give more careful 
consideration to the sequencing of subtasks 
within major topics or subdivisions of a chapter. 

6 . Optimal instructional sequences can be derived 
using learning hierarchies validated from test 
data . 



Recommendations 



The results of the present study suggest that sequence 
effects the overall efficiency of the learning process. 

The fact that no differences in immediate achievement were 
found between the random sequence group and the other hier- 
archically ordered grovq>s may have resulted from sample size 
and the complexity of the skills involved. The effects of 
sequence should be investigated by replications with more 
complex skills involving longer learning sequences and 
larger samples. 

BSjrther research should attempt to determine the effects 
of sequence upon the total learning process with students at 
different achievement levels in mathematics. That is, con- 
sideration should be given to the effects of sequence upon 
the attitudes and anxieties e:xperienced by learners in differ- 
ent sequence gro\:ps, the interaction effects between sequence 
and ability, and the effects of sequence upon immediate 
achievement, transfer, and long term retention. The effects 
of sequence upon learning mathematics shovild be investigated 
at both the secondary and elementary school levels. The 
effects of carefully sequenced instructional materials 
according to validated learning hierarchies on the performance 
of the slow learner and the remediational value of such 
instxoictlonal sequences should be investigated thoreughly. 
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STATISTICAL TABLES 
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Table 4. 



Correlation matrix for eulater I and the hieraroh* 
ioal ordering* 



1 

2 

3 

4 



_1 2 3 » 
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Tablt 6» Oorralation matrix for oluator III and th« hlar* 
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Tsxtbook ordsrlng 


•62 


AXAS approaoh 


.87 


Randon ordaring 


.61 



I 




I 



1 

t 

I 



Table 17. 



Proportion of sub^eota obtaining aooop table ooor 
on Pretaot I (N «■ 142s 24 point teat). 



es 






Aootiptablo Proportion of 

soore obtaining score 



24 


.23 


23 


CO 

r-f 

• 


22 


. 24 


21 


.11 


20 


.11 


19 


.05 


18 


.04 


17 


.04 



ivlean eoore cn 
Pretest I 



21.84 



t 



Table 18. Proportion of subjects obtaining acceptable ecores 
on pretest II (K •• 142 i 11 point tost). 



Acceptable Proportion of ^ Mean soore on 

score obtaining soore Pretest II 



0 

1 

2 

3 

4 



.71 0.563 

.12 
.09 
.04 
.04 



63 



67 



Tabls 19# Rtllabillty of noaoursmonto* 



Toot 


KMO 


Hionurohloal anolyoos 


•81 


AohiovoBont 


.94 


Tnuisfor 


.91 


Rotontion 


•93 



64 



o 

ERIC 
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Tabla 20* Idanti fixation of groups* 



Numbor of 
group 




Oesoriptlon 


1 




Logioal ssqusnoe 


2 




Guttnan Tsshnlqus 


3 




Random 


4 




Item difficulty 


5 




Phi coefficient 


6 




Textbook 


7 




AAAS approach 



Tabla 21c On«MWiiy analysis of varlanoe on aehievsiisnt* 



Seuroe of 
Variation 


MS 


df 




F 


P 


Treatments 


18*83 


6 




1.45 


.1992 


Experimental 

error 


12*98 


135 










Group Means 








1 2 


3 


4 


5 


6 


7 

« 


6*23 4*84 


6*47 


5.21 


6.05 


5.14 


7.62 



65 




69 



Table 22, One-way analyela of \-arianoe on tranofor. 



Source of 
Variation 


r/B 


df 


F 


p 


Treatments 


11.30 


6 


1*43 


.2051 


Experimental 

error 


7*88 


135 








Group r^eans 






1 2 


3 


4 


5 


6 7 


5*05 4*63 


5.26 


3.95 


3.6? 


3.57 5.24 



Table 23* One-way analyala of variance on retention* 



Source of 
Variation 


MS df 


F 


P 


Treatmente 


33.61 6 


2*12 


*0542 


Experimental 

error 


15.8if 135 








Group means 






1 2 


3 4 


5 6 


7 


7.50 7.32 


6*89 5.37 


7.62 4.95 


8.52 




66 



70 



Tabla 24# Time required to coaploto Inotruotional prograim# 



Group 


minimum 


Time in minutes 
maximum 


range 


1 


70 


205 


135 


2 


70 


120 


50 


3 


67 


191 


124 


4 


66 


230 


164 


5 


75 


230 


155 


6 


87 


230 


143 


7 


65 


230 


165 



Table 25* 0ne»way analysis of variance on time. 


Souree of 
Variation 


m 


df P 


P 


Traatnente 

Experimental 

error 


2931 • 20 
1498.99 


6 1.96 

135 


.0757 



Group noana 

1 2 3 4 5 6 7 

103.86 101.95 114.16 126.11 135*33 117.24 122.86 



67 




71 



Table 26. One-way analyaia of '/arieinoe on achievement 
(partial credit). 



Source of 
variation 


VS 


df 


F 


P 


Trcatnants 


43.21 


6 


1.688 


.1278 


Experimental 

error 


25*60 


135 








Group means 






1 2 


3 


4 5 


6 


7 


7.^2 


' 


7.05 8i67 


7i61 


11.03 



Table 27. One«»way analyaia of valance 
(reducing dleregardedf. 

V_ 


on achievement 


Source of 
variation 


\ 

MS 


df 


F 


P 


Trcatnenta 

Experimental 

error 


71*43 

42.12 


6 

135 


1.70 


.1259 




Group means 






1 2 


3 


4 5 


6 


7 


11.91 9.68 


13*05 


8.47 11.00 


11.10 


14.05 




68 

72 



Table 28 e 



Ona-way 

credit) 



analysis of n^ianoe on transfer (partial 



Source of df F P 

Variation 



Treatments 19#97 6 1.89 #0870 

Experimental 10»59 I35 

error 



1 

6.27 



Group means 

23^5 

5.89 6.89 5.00 4,76 



6 7 

4.52 6.66 



Table 29. Cbie«iiay analysis of variance on transfer (reducing 
disregurdod). 



Source of MS df F P 

Variation 



Treatments 31*^3 6 2.17 .0465 

Experimental 14.31 135 

error 



1 2 
7.05 6.84 



Group means 

345 
8.26 5.95 5.10 



6 7 
5.19 7.86 



69 




73 



Table 30* 



One-way finalyeis of varianoo on retention (partial 
oredl^ • 



s<^.irae of 
Variation 


VS 


df P 


P 


Troatnenta 


56.32 


£ 2.14 


.0525 


Bxporimontal 


26.35 


135 




error 










Group aeana 




1 2 


3 


4 5 6 


7 


10.91 10.26 


9.70 


7*84 10.81 7.57 


12.10 



Table 31* Onev^nay analysis of varianoo < 
ing disregarded). 


on retention (redue* 


Seuroe of 


US df 


P 


p 


variation 








TreatBonts 


95*08 6 


2.25 


•0416 


Bsperlnontal 


42.24 135 






error 










Group means 






1 2 


345 


6 


7 


14.23 12.84 


12,53 9.68 13.90 


10.00 


15.48 



o 

ERIC 
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APPENDIX B 



1 

I 

I 



1 



HIERARCHY FOR THE COMPUTATIONAL SKILLS OP 
RATIONAL NUr/BER ADDITION 



TERMINAL TASK 




O 

ERIC 



71 



75 



IDENTIFICATION OF HIERARCHY CELLS 



1. Adding with two fractions having like denominators 
where the siim requires no reducing. 



2 




2, Adding with three fractions having like denominators 
where the sum requires no reducing. 



1 

TO 




3. Adding a rational number named by a fraction and a whole 
number. 



1 

7 



4. Adding with two mixed numerals having like denominators, 

3 4 



5. Finding equivalent fractions by dividing both numerator 
and denominator by the same number, 

6 .□ 

5 “ T 

with fcW0 t'metions wi14\ Uke denominators where 
tins aum to lowest terms. 



h 




72 




7. Adding with two mixed numerals with like denominators 
where the eiim of the fractional parts requires reducing 
to lowest terms. . 




h. Adding with three mixed numerals where all denominators 
are alike and the sum of the fractional parts requires 
reducing to lowest terms. 




9. Changing names from an improper fraction to a mixed 
numeral. 



10. Adding with two mixed mnnerals where the denominators 
involved are alike and the sum of the fractional parts 
is greater than 1. 



11* Adding with three mixied numerals where the denominators 
involved are alike and the sum of the fractional parts 
is greater than 1. 









73 
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JirrjiiMJjXA 



PROGRAMMED INSTRUCTIONAL SEQUENCE 

ADDITION OF 
FRACTIONS 



PROGRAMMED 

SKILL 

BUILDING 

SEQUENCE 

7 



NAME ■ 

SCHOOL - 

71*78 ‘ ‘ 




I 

I 

I 

I 



Lesson 1 

1, Study this picture >. 

(s) How asny parts of the ssbmi sise are there? . 

(b) How Banj parts are shaded? — . 

(o) What part of the whole shape is shaded? 

(d) A naas for three fourths is 

k 

2 The 3 tells how many parts we are using o 

4 The 4 tells the nunber of parts in the whole <> 

The nuseral X called a fraction o 
4 




I 

ANSWERS 



U) U 

(b) 3 

(c) 3 
4 



24 -^ numerator 
4 f-4 denosdnator 

Each fraction is made up of two numerals which nams whole 
nniribsrse The number named above the bar is called tbs 

- The nudber named below the bar is called the numerator 

• denominator 



2. What part of each shape is shaded? 



(•> 




(b) 









(d) 





3. Hams the numsrators 



U) I 

2 



(b) 2 

4 



Name the dsnomlnatorh . 



(b) 1 
5 



s 



(c) 2 

8 



(e) 4 

7 



o 

ERIC 



n 



(a)l (b) 
2 



(o) 1 (d) j5 
4 6 



(a) 1 (b) 3 (c) 5, 



(a) 9 (b) 5 (c) 7 



(a) 2 
9 



Cilice 







AJJSvJExii 



4 « Cogqjlete the table 

Traction read 

(a) ^ two thirds 

3 

(b) i 

8 

(c) U 

9 

(d) 1 

2 one half 

(e) 

three fourths 

m 

flwe ssTsnths 



5« Hoft? can \n add? 

stoi of i, and 

.8 8 




8 



Use tha plotursa 




2 

8 



help you find the 







6. Use tha ahadsd portions to help you find ihsss subb« 





(b) five eighths 

(c) four ninths I 

I 

(•) 2 f 

4 

<"> I f 

f 

T 

I 

J 

i 

j 

(«> 4 i 

J 

6>) a 

5 



I 

f 

[ 



7» Usa tha number line to help you find theee exmo. 

(a) 



S. 



(b) 




^ ^ ^ 4 ^ 



4-> 1+f 
Jt u u 







^ » Ty 1 ^ ■■ 



8* What have we diaoovared? Whan the denomiriiatQra are the 
sane, we tuld the nuneratorao Me keep the aana danosilnator > 



Kwple 



4 

9 



*• k+lm LjLI - LaisJ 



9 9 




9^ Find the subb. Reneobar! When the danoolnatwa are the 
i, m add the nonaratora and keep the aana danaadnator> 



(•) 2 (b) 1 (e) 2 (4) 2 {•) L. 

9 7 8 6 L/ 11 



4 

8 



£ 

6 



1- 

11 



77 



O 
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(a) 2 
4 



(b) 5l 
7 



(a) 1 (b> 1 

9 7 

(c> Z (d) 2 
8 6 

(«) 2^ 

11 



81 



Lasscm 2 






1. Use tha shaded portions to help you find the subia 




I 



2 

6 




2. Use the aiadjer line to help you find the euiiei 



(a) 




% K c #4 It ^ IT ^4 * It /4 ^ K /4 /4 It 



1 

I > I 4 1 > 

1 Zt-iLtlJi 4' 



fe'irfe- H 



(b) 




(a) U 
16 






78 



82 



0»hs 








ANSHEilS 



O 
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3 * How do 70U Add wtih ^wo JVAOtions hAvljig nv* dMtaBliiAtoro? 
Add the -. and loeep the 



4* Do jou Add the bam way with tfaroo ArAotlons hAwlng 
denoAdnAtorA? 

K»apU t 2 # A + 1 - (2 -f II) 1 

9 9 9 9 

-q^.a 

9 9 



4 

9 



□ 



9; FIjmI the aim8« 




(a) 1. (b) 


2 


(c) X. 


10 


9 


13 


2. 


4 


2. 


10 


9 


13 


4. 


1 


6-. 









(d) g 

7 

1 

7 



(e) 1 
8 

2 

8 




10 



79 



mDBBrii!,tara 

danoaj.nator 

7«e 

6 , 7 



7 

(a) 7 

<b) 2 
9 

(c) 

13 

<d) i 

7 

(a) 2 

8 



83 



Tiprjacn ? 



How do you add 2 and 2? Uoo the plcturee to telp. 

8 




Doee ft _ 1 7 

8 ■ 

In the picture above we have two wholea and 2 another. 
Doea i + I - 2|? 

2 c Dee the pictures to hel^x'you find the sum. 




Does 



2 ^ 



2 + i? 

4 




8o 



84 



HM 








A numsrAl like la oalled a nmneraX - Cor you 

guasa why? 2^ 1« read "two and one fourth <>" 

3 ' What hawe we diaoovered? The nunber repreaented by 2 -t- ^ 
la the aane aa the nui^r repreaented by the odxed nuusral 

(a) What nijoed numeral namea the aama number aa 4 

(b) Doea 4^ *• ^ 

4« Complete the followi ngs 

vm 

5 



5- 



(a) 


3|-3 + 


(b) 


^-a + 


(c) 


4-, 


(d) 


4 + |-i 


(e) 




(f) 




Addt 




(a) 


2 



(b) 4 



1 

JL 



(e) ^ 

Jk 



1 

-1. 



(f) a 

3 



(e) 8 



a 

.9 



(g) JL. 

13 

ja 






81 

85 



(d) 11 



I 

HSi 



(h) 



18 






yea 



(wo ioay "four 
and five alxthc") 



I 

9 



5 + ^ 






(a) 4 (b) ^ 

(c) 8| (<») wjj 



(e) ^ (f) ^ 

(g) 12^ (h) 3^ 



ANSWEilS 



Leason 4 

lo Mra. Jcmea ha« 2 |> appLs' piea and charry pies 




(a) Hour nuiQr wbola plea does she have? 

(b) Mote that each email piece Is ^ of a pie , How 



■snj pieces of thia else are there? 



2 ^ Heir can wa. find how many pive Mrs ^ Jones has in all? 
Stu4]r the pletures. 




(a) 3 

(b) 4 



(a) First, add the whole pies, 2 + 1 - 

(b) Next, add the pieces of pie ■> 1 -f « 

5 5 

(o) Mrs, JoQss has whole pies and 

— ■ of anothsr pieo la all she has 






pies. 






(a) 

(b) 



4 

5 



(c) 3 

4 

5 



82 



O 

FRir 



86 






3. ReasnAMrt 2^ - 2 + ^ and “ 1 

Thus, 2 l - 2 + L _1 
^ 5 



+li - 1 I 




5 5 



Mrs, Jonas has piss in all. 

(a) Haas tbs sum rsprsssntsd in each shaded box. 



3 

+ 2 


fyt 


■ 

+ 


ii 

s:L£ 


1 

9 

L 

9 


i 


^ TT^ 



»oss ^ -i- - s|? 

(b) Nbat bass ws diseovsrsd? To add with two aiasd 
maMralSy first add tbs _ Second add the 



■■■ ■■ nonfasrs, 
Bnaplet 

+4 



9tsp 1. 2 Step 2. 4 Step 3. 9 




7 




o 

ERIC 



83 




3 

1 



4 



5. 5 



yes 



fractions 

whole 



5 

9 




CD 5 (2) 9 (3) 9S 

7 



ANSWERS 



4* Find the sia* 

(.) 4 






(<0 uf 

_1 



(b) ^ (o) 6| 






<4 



(e) 14^ (f) 10 ^ 






(«) 3 | 

(b) u|o 



(e) 1C& 
9 



(d) 10^ 



(.) 2985. 



if) 15 ^ 



84 

88 



o 



LtsMn 5 




1. Study these pictures o 

(e) What part is shaded? 



(b) What part is shaded? 




(c) Does 2 and represent the same part of the region? 

4 2 

■ ■ Does 2 a. i? 

4 2 

(d) Does 2r2 - i ? - 

4<f2 2 

2 and 1 are called ffmH,vflfnt fractions . They naisB the 
4 2 

the saas nuaiber. 

2. (a) Hoir aany eighths are shaded? 




(b) Haw aany sixteenths are shaded? 




(o) A fraction equl'valent to ^ is 



(d) Another Araeticm eqaiealent to 

(e) Does 8 H 1 ? _ 

16 ^* 2 

What have we dlseovered? 



The nuiriwr the fraction represents does not change when you 
divide ^nA _ tar the 



sans nueber. 



85 

89 



(a) 2_ 
4 

(b) 1 
2 



yes 

yes 



yes 

(a) 4 

(b) 8 



(c) 4 

8 

(d) i 
2 

(e) yes 



nuwrster 

denoednator 



ANS’rt'EP.fi 
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3. A fraction ia in Iowa at terms if there ia no whole 

ntimber other than 1 that will divide both the numerator 
and denominator. 

Examples: (a) ia in loweat terma since no idiola nuflAier 

U 

other than 1 will divide both the numerator 1 and the denoninatoi 



if. 



(b) 4 ^ loweat terms since 2 will divide both 

8 



the. 



.6 and the 



.8, 



iti.l . 

8*2 U 

r 

Are and 2 o<lui valent fractions? 

8 4 



Is 2 lowest terms? 
if 



Is 2 lowest terms? 
9 



4 . Find the missing numeral for each to show how to renama 



5. Mrite th ese f ractions in lowest terms 

(«) 4.0 

8 2 



(.) 

18 6 



( 0 ) 4 

8 



6 



(“J a-.Q 

“ o 



86 

90 



(b) numerator 
denominator 



these nuadwrs 


in lowest terms. 








« 2 (b) ^ t 


■MMBPWMW nm 




itH 


3 10-r 


□ ^ 


(a) 2 


□ 

0 


.mg 




(b) 5 




1 ’ 




(c) 3 1 



yes 

yes 



no (Since 3 will 
divide both 3 anc~[9 



(a) 1 

(b) 1 

(c) 4 

(d) 1 
4 




(•) i.. 

10 



(f) 2- 
9 



ANSWERS 

(®) 1 (f) 1 
2 3 



(g) 






(h) SL 
12 



(g) i (h) 3 
4 3 



(1) J5I . 
20 ' 



(J) 2k 
30 



(1) :l (J) k 
U 5 



Oc) 3 ^l 
48 



(1) IS. 
32 



(k) S (1) 

4 16 



o 
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Lasson 6 

1, Ho».' do w6 add 2. k3 When the 

8 8 



ar« the saioe, we add the 
We keep the sane . .. . 



So, 2 ^ 4 
8 8 



OiO. 






Ib 2 lowest terms? 
8 



2, Let's add 2 2» Use the picturea to help you find the 

8 8 



sum. 






1 

6 



8 



2 H 

8 

Is 4 in lowest terms? 
8 



What ftaction is equivalent to 6 (Use the picture to see the 

8 

answer)? 



2 is the siiq>lest name for 6. 

4 8 

If your best friends were named Laura Melinda Williamson and 
Daniel LsRoj Applegate, you would not caJJ. them by their full 
names. Instead you would simplify their names to Laura and 
Danny ° When we add with fractions, we will name the sum by 
its sin^last name. We write the answer in -- 



88 



92 



denominator? 

nuiiKsratore 

denominator 



3 , 4 ;, 2 

8 

yes 



I 



No, since 2 wiHi 
divide both 6 
and 8. | 



2 

4 



lowest terms 



n 





t 



ANSWEKy 



Thus, 

1+l^fL 
8 8 8 




3. To add with Araetlons having like denonlnators: (1) 

add the nuaerators (2) keep the sane denoainator, and 
(3 ) writs answer In ■ . 



Exaa^le: 3+5 ,8 

iS 16 lJf> 



I6i>8 2 



h 



in lowest tsnn is They are 



Add. RsBsaberl Ifarlte answers In lowest terms. 



(a) 1 

9 



(b) 1 
8 



.fractions. 




9 9 



"□O 



(c) 2- 
10 



(d) L. 

15 

2. 

-li. 



Jl 



(f) n 

24 

a. 

_24. 



89 



o 

ERIC 



93 



2 , 3 . 

4 



lowest terms 



equivalent 



(a) 3,3 

(b) 1 
2 

(p) 4 
5 



(d) 4 



(«} .4 

4 



(f) 2 

8 



Lasson 7 



. Let' 


B add 


‘l“>* 4- 


(a) 


Does 


4" 


4 + 1? 


(b) 


Does 


4- 


5+1? 


(c) 




4+J 


8 




4- 


5.J 


□. 




t 


9 


8 



(d) la there a sio^ler nans for 4? 

8 

(e) Ifarlte 4 in lowest terns » 4 « 

8 8 

30, 4 

<1 



2. 



Adc.o tfrlte answers in lowest teroso 







(b) 17^ 




(«) ai 
2^ 




90 

94 






(a) yea 

(b) yea 



I 



(c) 3, 1, U 



9|] 



(d) yes 



(e) 1 

2 



I 



9i 



I 

I 



(b) 2 



(c) 3^ 



I 



M 2 . 2 ; ; J 






. dp 
1 



a* • 



■ { ; . 

••V. '•• 






(d) 3^ 



(•) ^ ' 



AN.V'JPrs 

(d) 4|. 

(e) ^ 



I 

I 

<« IjJt (g) ^ 



(f) 4^ 





(g) nj 



i 

1 




91 



95 



Le 0 aon 6 



ANSVflSRS 



1. IbuBB thtt suaio 




U 



2 , 2 



9 





I 



i 

I 

I 

I 

f 

I 

J 

1 

I 

I 

I 

I 

I 

I 

r 

I 

1 



(b) ll«Be tb» mm raparetentad in aaoh atuidad box. 




2. Add. Ifrita anawtra In lonaat taraa. 

(b) ^ a 

4 it 

3 k 



.27 




mW 



a« 

i'di 



v< 

\v/: 



-r'lt 

» • #a 

iS 



r;*6i' 





ANSWESS 



9, 3, 23 




(a) 3 
(b) ^ 



93 



(c). 23 I 




(d) 




(o) 14 ^ 

’I 



(f) 



^ 18 

• 18 

13 1 . 
18 



(«) 6fj 

2 

52 - 
^ 15 



(«•) »fe 




(i) 3 ^ 




94 

98 - 



(0) liO I i 

(d) 31 1 

' r 

(•) 34 J ' 

1 

(f) 60| j 

(«) «| J 

<h) 34§ I 

I 

(1) 17 L I 

I 

“ 1 

i 
I 
I 
I 



L«i 0 on 9 



AHSUBSS 



DM tlw pieturvs to tolp jou answn* tb»M i^^i^oetlaaaar 



U) 




How aanj thirtis ero shodwlT 



3 



(b) How Beagr ^ boMo in 1 booct 
((o) J + ^ + ^ - I w j I . 



3 

3 



(d) Ooow I - 17 



(•) 



(D 



ra How manj sixths sre shsdsd? 



JPOS 

6 

6 



2. - Ceaplsts ths followlago 

(•> t-Cl (b) ^-Q 



(o) m* ^ 
□ 





Wist Itsvs ws diseovsrsd? Wisu both ths manrstor sod the 
dsiifrtnstor of s Arsotlon srs ths ssas, ths firsotion lusnil 
ths iradbsr 

3<f Jill has ^ esadjr bars. Dm ths shsdsd portions to help 

TOO find s sUnpIsr nans for 2 , 

• k 



f 1 



(a) Bow naagr fourths are shsdsd rsd? • 

(b) Pbur can^y bars Mdos ' . oainly bar, 

(e) How muly fourths srs shsdsd crssn? . 

(d) B«r Mflir fourths srs shaded In all? 

95 

99 



(a) 1 (b) 1 
(o) 2 (d) 5 

(s) 8 (f)15 
1 



4 

one whole 

3 

7 





(ANSWERS 



(*) 2 M ilb ^ 

k U k 



(f) Jin haa 



2-xQ.. 

U-. 4 



cdKiy bant. 



1, 3 

three four the 



o 

ERIC 



4. Uee tto shaded partione to tolp yoa urlto a sdatad nuseral 
that naaae the eaate maiber aa ^ . 

8 




li 

8 8 



□.Q 

8 8 



5* Caaq)le& the following 



(») 2 
7 



(b) + 

9 9 9 



tenas? 



j.Q -09-09 

OE? “ 



2. in lows at 
9 



j-Q 

9 3 



Thu e 12 



«. J-I-P-OP-OP 



(4) a_ + ___- 

• 1 

6> dee the ahadad porticaa to be]|> you write a idaad numeral 
that aamea the aame number aa £ » 




i-d.n.d 




4 ^ ^ 4 ^ 4 
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. 3 . 4 



(k> a, 1 , 2 , 1 * 



(b) 3, 4 



no, 1 



1 

1 

I 

i 

I 



(e) 2, 1, 2, if I 



(d) 2 . 2> 
7 7. 



4, 4, 1 



1, 1, 2, 1, 2 







I ANSWERS 



7* Wrlto nlxwl nuamral* for each* RaiMRsbarl Write 
ftraetiooaJ. part to loweat taxms. 



(b) 







(d) 

7 

(•) 



..Xf 

1 . 1 . 4 
4 



U> U. 

k 



’I 



(s) IS. 

12 






(b) 2. 

5 



5 
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L«tton 10 

1, tos tilt evB, 

(•) 3i 



8 



(b) Hots that the Qiwarator of tha firaotlon )JL 

7 



than tha 



. Fk^otiOQa of thia 



(c) 

(d) 



tjpa ban ba rananad aa . 

-oc;i 



2 . 



9 ia a alaplar 
Coaplhta tha following 



for 8 ^ 



“I 



aa a ataad nunaral 
luoa to lowaa^ taraa 







(a) 


11 


(b) 


gt9mtmr 




danoainator 




■ixad 




mawrala 


(e) 


k, k 


(d) 


k. 9 If 



17, 

17, 17, 1 



18 ^ 
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4. Add. Write anBiMra in eioplieat fora, 
(a) 15 ^ 



n 



it 






19 + 1 

15 15 5 



(b) sj 






a 



12 12 + 
4 



(o) 




O 

ERIC 



99 

103 



20 

6, Z, 3 
39 



40 i 



(a) IB. 

1 , ; 



(b) 4, 



(e) 24. 



3 



1, 13 



11 , 2 



Laston 11 

1. Maa» the eun repreoentad in each shaded box* 
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I AMSMERS 



7 , 2 , 

12 



1 1 + 12 - „ 



Thus 




4 

1 



I 

5 

4 

5 



13 




2* Add. Urlta answers In l0west<>tsms. 

(•) 13 ^ 



’i 



OP- "0*^9 



13 



g 

5 



2 



(•) 25 » 
25 

P 



26 U 

12 
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TESTING INSTRUMENTS 
RATIONAL NUMBER ADDITION 



Najne Grade 

Teacher School 



Add: 

1 2 1 

1. T" 



2 . 



1 ' 



iiy 



2 

nr 

2 

~r 




-i- 




2 






7 



2 

“ 5 " 



10 

3 



o 
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5. Write these fractions in lowest terms: 



6 


6 


5 _ 


-gr - 


"TT" " 


TT) - - 


Add. Write answers 


in lowest terms. 




6 - -fe 


-h 


2 

“TO 




2 




7- 4 ^ 




13 1 




3 h 




8. 3 ^ 


h 


22 ^ 




11^ 








T? 



9* Write mixed numerals that stand for the same number as 
each of these fractions. 




16 

“T = 



o 

ERIC 
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109 







Add. Write answers in lowest terms. 





2 



7 



T3 









9 

10 

11 



11 

I? 
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110 
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ratio: f.M. I'KIT/ITiRR ADDT'j'I'n 

Pretont I 



Name 



Teacher 



Grade 



School 



1. Circle the numerals that are fractions. 

1 ^ ^ 



2 . 



Circle the pictures which show thirds. 




3. ilatch each numeral with the shaded part of a sliape as sliown. 





4* V/ritc the mimeral that naxies three eighths, 

5. V/rite the v/ords that name ^ 

6. Fill in the | | to make these sentences true, 

7. Add, 



4 


21 


14 


39 


605 


5 


+ 5 


4 27 


4 86 


3 


+ 3 








4 74 



8. Divide, 

4 + 2= 9 + 3= 12 + 4 

18 + 3 = 42 + 3 = 54 + 2 
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H/iTIOl'TAL AODITIOH 

Pretost JI 



Name 

Grade 



Add. 




Teacher 

School 




5. Reduce to lowest terms: ^ = 



Add. Write answers in lowest terras. 

6. f 7. 9^ 

5 2 



9* Write as a mixed numeral: ^ = 



Add. Write answers in lowest terms. 



10 . 11 



+ 15 



11 . 12 
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RATIONAL mjrrBER ADDITION 
Achievement Test Form A 



Name Teacher 

Orade School 

Add; V/rite answers in lov/est terms. 
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F 






2 

JL 






4 

IF 

2 

IF 



3. 



8 




3 

9 



11 

I? 

8 
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6 . 2 



1 

F 





no 
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1 

TI 



4 

TT 

5 

TT 




13 . 




14 . 




16 . 



15 TO 




o 

ERIC 
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RATIONAL T'TUf.'ffiER SUBTRACTION 
Transfer Test 

Nam© Teacher 



Grade School 

Subtract. Write answers in lowest terras. 





5. 



9 



7 





7. 8. 5 I 

1 

~ Tg - 2 



10 . 



15 



12 

T3 



- 6 
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RATIONAL NTT/IBER ADDITION 
Achievement Test Norm B 
(Retention Tost) 



Name 




Teacher 


Grade 




School 


Add. Write answers 


in lowest 


terras. 




2. 








-ii. 



4. 10 


5. Y 




if 


h 


f 




6 . 



6 



1 

I? 






> 



4 
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9 . 



7 



3 



7 



2 
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10 . 5 
13 
17 
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T 

1 

7 

6 

7 



11 . 



2 

F 




12 . 



5 

IS 



13. 11^ 14. I 







16. 4 
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14 



2 

9 



18 . 
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